-- Logs begin at Tue 2022-05-24 09:17:13 UTC, end at Tue 2022-05-24 11:05:26 UTC. -- May 24 10:04:11.574867 dv-cinder-843118-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 24 10:04:11.583584 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 10:04:11 2022] *** May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: nodename: dv-cinder-843118-2-iscsi May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: machine: x86_64 May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: clock source: unix May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: pcre jit disabled May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: detected number of CPU cores: 4 May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: current working directory: / May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: detected binary path: /usr/bin/uwsgi-core May 24 10:04:11.585574 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: your processes number limit is 47680 May 24 10:04:11.590599 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: your memory page size is 4096 bytes May 24 10:04:11.590599 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: detected max file descriptor number: 2048 May 24 10:04:11.590599 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: lock engine: pthread robust mutexes May 24 10:04:11.590599 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: thunder lock: enabled May 24 10:04:11.590599 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 24 10:04:11.590599 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 10:04:11.631358 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: Python main interpreter initialized at 0x55fec5f8b9c0 May 24 10:04:11.631358 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: python threads support enabled May 24 10:04:11.631358 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: your server socket listen backlog is limited to 100 connections May 24 10:04:11.631358 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: your mercy for graceful operations on workers is 90 seconds May 24 10:04:11.631722 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: mapped 403077 bytes (393 KB) for 2 cores May 24 10:04:11.631799 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: *** Operational MODE: preforking *** May 24 10:04:11.632037 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: *** uWSGI is running in multiple interpreter mode *** May 24 10:04:11.632037 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: spawned uWSGI master process (pid: 103798) May 24 10:04:11.632245 dv-cinder-843118-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 24 10:04:11.632788 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: spawned uWSGI worker 1 (pid: 103799, cores: 1) May 24 10:04:11.633301 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: spawned uWSGI worker 2 (pid: 103800, cores: 1) May 24 10:04:11.633301 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 10:04:12.826235 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 24 10:04:12.826862 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 24 10:04:13.556063 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:04:13.556063 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: from cryptography.utils import int_from_bytes May 24 10:04:13.556412 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:04:13.556412 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: from cryptography.utils import int_from_bytes May 24 10:04:13.561996 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:04:13.561996 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: from cryptography.utils import int_from_bytes May 24 10:04:13.562349 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:04:13.562349 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: from cryptography.utils import int_from_bytes May 24 10:04:16.187192 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 10:04:16.187656 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 10:04:16.187963 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 10:04:16.188323 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 10:04:16.188579 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 10:04:16.188967 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.189315 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.189642 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.189962 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.190301 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.190615 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.190934 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.191252 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.191581 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.191842 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.191995 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 10:04:16.192383 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.192472 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 10:04:16.192557 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843118-2-iscsi {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.192776 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 10:04:16.192873 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.193079 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.193170 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 10:04:16.193414 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.193518 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 10:04:16.193732 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.193823 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.194037 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.194138 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.194358 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.194443 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.194663 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.194759 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.195078 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.195179 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.195395 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.195596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.195729 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.195906 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196028 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196216 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196364 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196555 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196696 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196869 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.196969 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.197178 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.197279 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843118-2-iscsi {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.197528 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.197674 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.197841 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.197933 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.198166 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.198257 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.199032 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.199155 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.199437 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843118-2-iscsi {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.199613 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.199785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.199955 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.200047 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.200273 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.200365 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.200761 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.200873 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.201080 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.201166 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.201414 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.201513 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.201732 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.201824 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202050 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202135 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202355 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202446 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202681 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202774 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.202999 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.203091 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.203315 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.203413 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.203632 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.203718 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.203934 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204019 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204244 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204358 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843118-2-iscsi {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204569 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204671 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204872 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.204990 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.205175 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.205292 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.205504 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.205701 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.205795 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206023 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206108 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206341 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206434 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206661 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206752 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.206982 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207080 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207233 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207336 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207481 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207610 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207797 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.207937 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.208118 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.208261 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.208425 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.208587 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.208731 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.208897 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209038 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209213 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209376 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209558 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209687 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209884 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.209984 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.210195 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.210288 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.210507 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.210593 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.210763 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.210881 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.211076 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.211219 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.211404 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.211497 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.211730 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.211825 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212059 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212157 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212317 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212481 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212574 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212733 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.212884 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.213039 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.213213 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.213402 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.213554 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.213720 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.213867 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.214045 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.214884 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.214982 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.215232 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.215474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.215571 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.215809 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.215905 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.216128 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.216214 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.216444 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.216530 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.216762 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.216851 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217064 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217153 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217336 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217426 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217639 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217739 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.217945 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.218031 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.218264 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.218350 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.218570 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.218755 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.218914 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219001 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219170 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219316 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219470 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219669 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219796 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.219998 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.220092 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.220314 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.220407 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.220641 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.220733 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.220949 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.221039 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.221282 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.221379 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.221600 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.221688 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.221914 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222010 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222243 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222335 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222574 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222660 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222884 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.222986 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.223141 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.223237 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.223449 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.223541 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.223760 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.223859 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224060 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224154 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224380 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224466 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224688 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224775 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.224994 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.225078 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.225315 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.225419 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.225631 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.225719 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.225933 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226032 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226242 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226334 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226545 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226629 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.226941 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.227156 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.227246 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.227453 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.227537 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.227759 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.227860 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228068 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228159 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228368 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228460 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228671 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228769 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.228973 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.229058 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.229322 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.229421 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.229596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.229698 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.229934 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.230176 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.230273 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.230683 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.230777 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231001 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231094 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231319 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231451 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231637 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231736 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.231949 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232041 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232258 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232388 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232565 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232660 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232884 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.232970 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.233197 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.233308 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.233553 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.233655 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.233862 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.233949 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.234182 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.234277 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.234508 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.234603 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.234824 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.234917 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.235137 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.235230 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.235449 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.235535 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.235762 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.235857 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.236071 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.236157 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.236383 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.236476 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.236692 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.236776 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.237001 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.237085 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.237329 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.237432 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.237657 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.237756 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.237985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.238078 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.238302 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.238395 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.238615 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.238701 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.238926 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.239025 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.239236 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.239321 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:04:16.239555 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.239641 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.239885 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.239991 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.240207 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.240292 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.240521 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.240606 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.240868 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.240980 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.241205 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.241345 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.241543 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.241636 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.241866 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.241959 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.242185 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.242275 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.242497 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.242584 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.242809 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.242904 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.243120 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.243206 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.243437 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.243529 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.243765 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.243860 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.244077 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.244161 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.244400 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.244485 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.244713 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.244807 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.245024 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.245111 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.245886 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.246019 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.246261 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.246506 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.246600 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.246847 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.246934 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.247189 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.247284 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.247504 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.247588 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.247816 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.247901 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.248134 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.248227 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.248444 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.248537 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.248755 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.248841 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.249065 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.249151 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.249399 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.249488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.249708 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.249793 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250012 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250321 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250406 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250626 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250721 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.250946 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.251032 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.251278 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.251369 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.251584 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.251677 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.251917 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252003 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252225 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252317 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252537 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252631 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252847 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.252939 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.253159 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.253245 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.253490 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.253588 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.253803 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.253895 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.254115 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.254200 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.254420 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.254517 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.254742 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.254834 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255051 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255137 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255356 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255454 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255672 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255764 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.255977 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256061 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256285 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256383 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256593 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256686 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256901 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.256985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.257211 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.257341 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.257551 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.257640 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.257864 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.257961 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.258182 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.258279 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.258495 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.258579 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.258808 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.258902 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.259122 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.259207 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.259440 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.259526 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.259749 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.259832 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.260057 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.260143 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.260375 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.260461 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.260689 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.260785 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.261579 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.261710 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.261893 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.262200 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.262285 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.262507 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.262600 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.262813 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.262904 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.263123 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.263235 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.263427 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.263555 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.263731 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.263872 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264035 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264204 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264338 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264525 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264643 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264847 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.264966 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.265169 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.265311 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.265521 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.265609 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.265855 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.265943 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.266178 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.266312 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.266500 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.266594 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.266811 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.266895 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.267150 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.267239 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.267469 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.267597 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.267788 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.267882 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.268111 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.268198 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.268425 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.268511 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.268738 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.268832 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.269048 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.269138 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.269384 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.269473 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.269698 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.269794 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270008 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270094 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270327 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270450 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270635 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270784 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.270946 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271040 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271254 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271339 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271565 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271659 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271878 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.271965 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.272195 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.272324 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.272500 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.272587 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.272810 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.272896 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.273122 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.273210 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.273460 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.273557 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.273800 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.273885 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.274116 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.274200 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.274426 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.274522 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.274739 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.274833 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275050 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275143 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275359 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275452 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275668 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275755 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.275979 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.276072 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.276295 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.276427 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.276627 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.276772 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.276946 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.277254 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.277389 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.277598 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.277732 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.277907 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.278070 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.278238 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.278397 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.278516 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.278729 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.278817 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.279052 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.279147 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.279642 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.279736 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.279965 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.280052 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.280297 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.280392 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.280611 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.280698 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.280918 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.281002 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.281229 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.281349 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.281571 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.281656 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.281878 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282023 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282184 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282272 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282494 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282587 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282806 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.282892 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.283114 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.283201 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.283423 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.283516 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.283727 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.283813 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284040 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284133 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284348 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284435 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284652 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284746 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.284955 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.285040 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.285279 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.285367 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.285606 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.285707 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.285916 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286008 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286236 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286322 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286560 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286653 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286863 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.286954 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.287167 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.287252 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.287486 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.287580 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.287792 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.287877 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.288096 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.288188 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.288399 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.288485 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.288702 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.288787 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289043 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289130 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289376 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289464 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289684 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289770 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.289992 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.290088 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.290308 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.290435 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.290616 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.290701 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.290929 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291014 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291239 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291326 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291549 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291633 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291866 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.291968 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.292223 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.292350 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.292618 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.292729 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.292937 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.293214 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.293363 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.293572 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.293659 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.293883 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.293968 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.294206 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.294301 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.294517 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.294604 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.294830 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.294916 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.295141 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.295275 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.295447 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.295539 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.295758 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.295853 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.296076 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.296162 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.296392 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.296643 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.296728 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.296978 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.297072 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.297320 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.297431 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.297643 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.297752 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.297966 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.298054 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.298285 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.298372 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.298610 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.298704 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.298929 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299018 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299249 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299336 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299560 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299655 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299871 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.299967 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.300178 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.300265 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.300483 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.300570 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.300792 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.300886 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.301096 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.301191 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.301427 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.301525 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.301733 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.301820 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302043 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302170 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302347 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302434 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302648 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302735 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.302949 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303043 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303252 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303339 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303566 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303652 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303869 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.303954 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.304177 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.304264 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.304479 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.304749 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.304837 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.305090 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.305188 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.305413 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.305516 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.305718 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.305804 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306022 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306108 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306359 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306454 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306662 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306748 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.306995 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.307081 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.307320 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.307414 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.307653 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.307747 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.308564 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.308653 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.308933 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.309092 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.309323 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.309425 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.309632 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.309751 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.309934 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.310068 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.310239 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.310383 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.310548 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.310689 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.310856 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311005 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311158 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311320 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311475 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311632 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311783 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.311957 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.312092 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.312256 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.312400 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.312576 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.312705 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.312886 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313014 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313204 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313347 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313542 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313663 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313870 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.313965 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.314194 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.314281 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.314502 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.314589 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.314812 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.314899 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.315122 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.315208 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.315424 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.315509 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.315733 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.315836 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316035 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316123 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316349 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316436 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316651 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316736 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.316955 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317040 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317256 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317370 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317585 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317683 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317891 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.317987 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.318192 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.318285 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.318513 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.318598 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.318822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.318920 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.319125 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.319212 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.319427 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.319525 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.319734 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.319828 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320038 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320340 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320434 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320647 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320791 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.320962 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321048 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321290 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321391 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321605 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321693 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321913 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.321998 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.322226 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.322354 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.322624 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.322845 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.322932 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.323160 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.323253 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.323541 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.323626 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.323905 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.324143 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.324347 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.324475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.324656 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.324795 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.324963 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.325124 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.325303 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.325467 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.325618 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.325785 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.325925 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.326098 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.326232 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.326416 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.326534 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.326734 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.326874 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.327045 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.327176 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.327376 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.327531 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.327693 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.327837 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328004 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328151 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328312 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328468 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328628 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328777 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.328938 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.329087 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.329246 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.329419 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.329585 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.329721 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.329913 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330035 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330233 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330343 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330542 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330653 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330857 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.330952 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.331167 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.331263 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.331476 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.331562 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.331783 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.331879 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.332095 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.332182 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.332403 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.332543 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.332709 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.332809 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333016 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333103 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333350 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333440 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333662 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333757 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.333967 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334059 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334276 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334361 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334585 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334677 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334891 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.334976 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.335197 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.335289 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.335507 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.335600 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.335816 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.335902 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.336125 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.336218 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.336435 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.336521 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.336741 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.336826 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.337048 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.337141 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.337378 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.337467 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.337685 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.337778 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338006 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338099 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338314 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338399 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338617 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338748 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.338925 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.339574 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.339890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.340048 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.340205 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.340419 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.340506 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.340742 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.340918 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.341079 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.341229 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.341428 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.341579 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.341751 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.341859 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.342085 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.342174 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.342419 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.342513 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.342753 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.342839 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.343062 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.343147 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.343366 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.343465 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.343684 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.343769 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344000 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344087 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344319 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344414 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344644 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344730 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.344954 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.345045 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.345256 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.345381 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.345587 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.345674 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.345909 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346002 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346211 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346298 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346513 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346599 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346818 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.346913 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.347135 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.347222 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.347437 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.347523 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.347756 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.347843 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.348072 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.348159 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.348387 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.348475 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.348705 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.348804 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.349018 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.349154 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.349368 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.349488 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.349682 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.349786 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.350105 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.350190 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.350420 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.350505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.350737 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.350832 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351038 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351123 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351350 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351435 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351659 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351753 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.351978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.352063 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.352297 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.352382 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.352606 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.352731 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.352916 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353029 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353220 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353359 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353566 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353679 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353888 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.353995 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.354201 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.354308 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.354522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.354614 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.355610 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.355713 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.355937 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.356202 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.356369 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.356527 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.356638 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.356846 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.356934 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.357160 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.357323 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.357492 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.357588 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.357805 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.357892 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.358107 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.358193 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.358412 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.358553 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.358733 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.358875 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.359040 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.359191 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.359352 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.359614 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.359709 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.359934 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360021 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360249 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360342 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360552 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360638 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360872 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.360964 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.361174 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.361308 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.361519 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.361607 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.361837 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.361934 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.362152 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.362246 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.362468 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.362555 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.362773 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.362859 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.363094 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.363181 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.363403 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.363488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.363706 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.363792 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364011 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364314 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364401 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364650 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364736 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.364953 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.365041 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.365347 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.365438 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.365728 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.365832 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.366047 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.366134 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.366357 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.366442 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.366689 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.366849 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367000 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367086 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367312 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367400 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367628 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367722 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.367938 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368024 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368239 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368325 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368542 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368636 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368852 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.368938 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.369155 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.369241 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.369483 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.369583 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.369787 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.369874 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.370093 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.370186 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.370458 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.370549 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.370815 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.371063 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.371189 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.371434 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.371531 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.371745 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.371870 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372067 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372160 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372375 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372470 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372680 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372766 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.372985 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.373066 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.373306 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.373408 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.373648 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.373735 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.373950 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.374036 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.374275 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.374362 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.374591 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.374676 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.374912 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375006 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375236 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375323 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375543 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375629 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375863 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.375950 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.376180 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.376267 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.376494 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.376587 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.376807 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.376901 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.377117 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.377205 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.377456 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.377544 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.377771 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.377878 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.378085 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.378172 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.378418 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.378548 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.378731 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.378873 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.379050 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.379189 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.379363 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.379507 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.379675 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.379987 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.380073 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.380301 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.380388 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.380611 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.380714 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.380920 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.381008 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.381229 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.381348 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.381569 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.381665 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.381879 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382011 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382186 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382272 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382495 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382595 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382804 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.382890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.383112 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.383198 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.383421 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.383507 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.383745 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.383831 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.384062 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.384148 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.384393 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.384480 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.384709 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.384797 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.385025 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.385114 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.385361 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.385452 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.385678 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.385774 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.386105 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.386199 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.386479 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.386665 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.386788 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.386999 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.387095 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.387327 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.387423 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.387650 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.387737 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.387975 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.388062 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.388299 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.388388 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.388612 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.388698 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.388924 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389011 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389235 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389394 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389571 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389658 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389885 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.389980 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.390198 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.390411 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.390520 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.390749 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.390855 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.391073 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.391167 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.391397 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.391546 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.391718 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.391807 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392039 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392127 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392359 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392586 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392673 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392901 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.392989 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.393232 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.393337 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.393580 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.393680 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.393904 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394001 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394218 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394304 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394534 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394621 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394858 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.394952 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.395163 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.395250 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.395466 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.395562 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.395776 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.395870 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.396084 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.396172 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.396387 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.396546 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.396706 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.396800 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397013 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397101 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397337 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397439 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397656 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397753 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.397962 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.398049 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.398315 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.398492 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.398632 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.398836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.398952 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.399153 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.399272 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.399476 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.399588 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.399800 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.399887 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.400115 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.400213 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.400421 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.400514 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.400740 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.400828 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.401045 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.401153 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.401372 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.402022 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.402330 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.402555 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.402701 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.402987 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.403079 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.403319 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.403414 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.403687 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.403851 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.403998 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.404177 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.404301 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.404784 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.404883 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.405125 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.405214 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.405485 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.405576 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.405810 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.405909 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.406117 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.406211 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.406421 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.406507 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.406730 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.406825 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407031 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407118 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407333 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407427 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407635 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407731 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.407937 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408024 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408246 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408333 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408548 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408643 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408855 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.408942 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.409158 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.409244 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.409493 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.409589 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.409797 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.410048 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.410136 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.410402 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.410498 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.410725 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.410819 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411038 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411132 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411341 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411440 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411656 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411743 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.411965 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412051 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412288 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412386 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412598 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412684 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412906 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.412993 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.413290 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.413424 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.413567 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.413655 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.413881 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.414096 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.414193 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.414435 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.414522 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.414758 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.414852 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.415090 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.415189 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.415403 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.415490 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.415713 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.415802 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416020 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416119 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416328 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416415 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416647 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416732 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.416956 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.417049 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.417827 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.417987 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.418139 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.418450 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.418597 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.418759 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.418846 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419066 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419159 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419375 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419683 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419778 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.419991 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.420078 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.420298 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.420392 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.420606 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.420700 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.420915 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421002 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421226 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421327 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421590 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421677 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421908 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.421996 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.422225 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.422390 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.422542 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.422637 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.422859 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.422953 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.423191 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.423279 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.423511 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.423597 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.423843 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.423942 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.424150 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.424243 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.424469 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.424554 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.424793 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.424888 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.425105 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.425192 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.425432 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.425693 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.425846 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.426037 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.426157 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.426365 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.426475 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.426704 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.426805 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427043 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427132 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427360 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427449 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427675 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427838 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.427990 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.428084 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.428308 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.428394 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.428618 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.428712 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.428933 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429026 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429251 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429366 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429590 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429687 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429906 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.429994 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.430208 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.430294 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.430513 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.430601 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.430814 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.430908 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.431119 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.431207 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.431433 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.431528 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.431738 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.431824 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.432041 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.432135 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.432347 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.432474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.432653 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.432748 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.433126 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.433223 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.433468 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.433657 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.433774 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.433986 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.434074 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.434315 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.434414 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.434637 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.434732 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.434947 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435034 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435249 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435336 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435546 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435640 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435845 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.435932 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.436145 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.436233 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.436449 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.436537 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.436749 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.436844 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437050 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437224 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437355 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437570 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437666 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437897 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.437986 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.438227 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.438319 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.438546 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.438634 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.438875 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.438963 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.439195 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.439289 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.439503 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.439599 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.439820 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.439907 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.440127 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.440213 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.440444 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.440539 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.440760 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.440847 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.441070 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.441156 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.441408 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.441516 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.441738 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.441838 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442045 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442140 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442353 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442442 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442660 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442746 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.442967 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443053 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443273 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443361 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443579 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443704 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443886 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.443973 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.444201 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.444290 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.444501 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.444587 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.444810 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.444905 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.445125 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.445219 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.445467 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.445554 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.445780 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.445882 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.446088 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.446174 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.446397 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.446484 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.446706 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.446802 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447011 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447317 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447410 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447622 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447717 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.447930 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.448024 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.448240 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.448327 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.448627 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.448727 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.449008 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.449210 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.449436 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.449568 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.449757 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.449888 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.450086 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.450234 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.450413 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.450578 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.450758 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.450904 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.451073 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.451235 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.451416 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.451556 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.451743 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.451889 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452058 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452208 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452374 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452538 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452673 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452869 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.452956 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103799) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 10:04:16.453191 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.453534 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.453875 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.454196 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.454517 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.454836 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.455161 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.455480 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.455798 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.456131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.456465 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.456780 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.457113 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.457469 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.457800 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.458122 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.458441 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.458778 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.459098 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.459415 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.459739 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.460055 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.460373 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.460698 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.461020 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.461453 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.461790 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.462106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.462438 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.462773 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.463125 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.463447 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.463795 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.464821 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.465401 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.465678 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.465997 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:04:16.466248 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103800) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 10:04:16.512279 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO dbcounter [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Registered counter for database nova_api May 24 10:04:16.526812 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO dbcounter [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Registered counter for database nova_api May 24 10:04:16.534126 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_db.sqlalchemy.engines [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103799) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.540791 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writer thread running {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.547183 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_db.sqlalchemy.engines [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103800) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.553645 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writer thread running {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.598954 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.context [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0b3c262b-14d7-4ecd-824a-690cdc4a9b4a(cell1) {{(pid=103799) load_cells /opt/stack/nova/nova/context.py:464}} May 24 10:04:16.603068 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:04:16.605190 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:04:16.606310 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:04:16.608256 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:04:16.610266 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO dbcounter [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Registered counter for database nova_cell0 May 24 10:04:16.614240 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO dbcounter [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Registered counter for database nova_cell1 May 24 10:04:16.614501 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.context [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0b3c262b-14d7-4ecd-824a-690cdc4a9b4a(cell1) {{(pid=103800) load_cells /opt/stack/nova/nova/context.py:464}} May 24 10:04:16.618627 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_concurrency.lockutils [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103800) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:04:16.620215 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_concurrency.lockutils [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103800) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:04:16.621936 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_concurrency.lockutils [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103800) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:04:16.622169 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_db.sqlalchemy.engines [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103799) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.623977 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_concurrency.lockutils [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103800) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:04:16.624724 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_db.sqlalchemy.engines [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103799) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.626058 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO dbcounter [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Registered counter for database nova_cell0 May 24 10:04:16.628844 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writer thread running {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.630116 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO dbcounter [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Registered counter for database nova_cell1 May 24 10:04:16.637830 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_db.sqlalchemy.engines [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103800) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.639081 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_db.sqlalchemy.engines [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103800) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.646268 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writer thread running {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.646613 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writer thread running {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.766926 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writer thread running {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.776914 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO dbcounter [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Registered counter for database nova_cell0 May 24 10:04:16.785887 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_db.sqlalchemy.engines [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103799) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.790188 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writer thread running {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:16.797041 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO dbcounter [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Registered counter for database nova_cell0 May 24 10:04:16.805703 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG oslo_db.sqlalchemy.engines [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103800) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:04:16.811074 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writer thread running {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:04:17.209011 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: WARNING keystonemiddleware.auth_token [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:04:17.215882 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: WARNING keystonemiddleware.auth_token [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:04:17.216492 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: WARNING keystonemiddleware.auth_token [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:04:17.223015 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: WARNING keystonemiddleware.auth_token [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:04:17.558920 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: WARNING keystonemiddleware.auth_token [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:04:17.564809 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: WARNING keystonemiddleware.auth_token [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:04:17.569258 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: WARNING keystonemiddleware.auth_token [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:04:17.570144 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fec5f8b9c0 pid: 103799 (default app) May 24 10:04:17.572208 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:04:17.573965 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] 10.222.0.35 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001733 May 24 10:04:17.574674 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 1/1] 10.222.0.35 () {50 vars in 788 bytes} [Tue May 24 10:04:17 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:04:17.576110 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: WARNING keystonemiddleware.auth_token [None req-643163c8-6c66-4276-b292-d485d88d50eb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:04:17.581460 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fec5f8b9c0 pid: 103800 (default app) May 24 10:05:38.523419 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:38.523419 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001540 May 24 10:05:38.523419 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 2/2] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:38.524441 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writing DB stats nova_api:SELECT=3 {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:38.525500 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writing DB stats nova_cell0:SELECT=3 {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:38.526523 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writing DB stats nova_cell1:SELECT=3 {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:38.528162 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:38.973326 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-05a7cee2-4087-4e7a-b213-68214b341f4e admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:39.003471 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-05a7cee2-4087-4e7a-b213-68214b341f4e admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.488635 May 24 10:05:39.004708 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 1/3] 10.222.0.35 () {60 vars in 1291 bytes} [Tue May 24 10:05:38 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:05:41.271510 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:41.272437 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-a02b15c1-1c73-4749-a8ac-1d132ddc9ce5 None None] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001416 May 24 10:05:41.272831 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 3/4] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:41.336329 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-50e7fcc0-e302-44f4-a131-16e713d9d870 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "ram": 256, "rxtx_factor": 1.0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "swap": 0, "disk": 1, "id": "c1"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:41.356699 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:05:41.356699 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: warnings.warn(msg) May 24 10:05:41.446482 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-50e7fcc0-e302-44f4-a131-16e713d9d870 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.167008 May 24 10:05:41.454553 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 2/5] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:41 2022] POST /compute/v2.1/flavors => generated 407 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:05:41.461435 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_api:SELECT=3 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:41.462485 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_cell0:SELECT=3 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:41.463845 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_cell1:SELECT=3 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:41.464622 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:41.672182 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-15e0426c-51b0-49e7-9a67-707f68c89a47 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:41.674677 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:05:41.674677 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: warnings.warn(msg) May 24 10:05:41.822179 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-15e0426c-51b0-49e7-9a67-707f68c89a47 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.365897 May 24 10:05:41.833380 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 4/6] 10.222.0.35 () {64 vars in 1335 bytes} [Tue May 24 10:05:41 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:05:44.112590 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-50e7fcc0-e302-44f4-a131-16e713d9d870 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:44.113718 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-50e7fcc0-e302-44f4-a131-16e713d9d870 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001567 May 24 10:05:44.114513 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 3/7] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:44.243115 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-cc5a855c-b3ac-4711-b9af-1595b9a0e1b2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "os-flavor-access:is_public": true, "disk": 5, "rxtx_factor": 1.0, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "ds512M"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:44.261706 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:05:44.261706 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: warnings.warn(msg) May 24 10:05:44.373997 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-cc5a855c-b3ac-4711-b9af-1595b9a0e1b2 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.253453 May 24 10:05:44.374549 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 5/8] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:44 2022] POST /compute/v2.1/flavors => generated 404 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:05:44.387361 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-4b3805da-f482-42d5-93bb-6985bd96d9b5 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:44.388932 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:05:44.388932 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: warnings.warn(msg) May 24 10:05:44.495167 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-4b3805da-f482-42d5-93bb-6985bd96d9b5 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.112343 May 24 10:05:44.495782 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 4/9] 10.222.0.35 () {64 vars in 1335 bytes} [Tue May 24 10:05:44 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:05:46.751627 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-cc5a855c-b3ac-4711-b9af-1595b9a0e1b2 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:46.752647 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-cc5a855c-b3ac-4711-b9af-1595b9a0e1b2 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 May 24 10:05:46.753042 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 6/10] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:46.816357 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-0692458e-5aa4-46f8-87fc-e4d9635adcf0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 1024, "id": "d2", "name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 10, "rxtx_factor": 1.0, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:46.981835 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-0692458e-5aa4-46f8-87fc-e4d9635adcf0 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.222072 May 24 10:05:46.982535 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 5/11] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:46 2022] POST /compute/v2.1/flavors => generated 404 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:05:46.995673 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-349f67b7-6ff2-42d8-a6b2-11b48d5d778c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:47.113510 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-349f67b7-6ff2-42d8-a6b2-11b48d5d778c admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.121894 May 24 10:05:47.114090 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 7/12] 10.222.0.35 () {64 vars in 1335 bytes} [Tue May 24 10:05:46 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:05:49.403072 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-0692458e-5aa4-46f8-87fc-e4d9635adcf0 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:49.403874 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-0692458e-5aa4-46f8-87fc-e4d9635adcf0 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001951 May 24 10:05:49.404451 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 6/13] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:49.466356 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-d2b27859-0d30-40e3-9e3a-e0f84339c3b5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 2048, "disk": 10, "os-flavor-access:is_public": true, "vcpus": 2, "id": "d3", "rxtx_factor": 1.0, "name": "ds2G"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:49.561095 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-d2b27859-0d30-40e3-9e3a-e0f84339c3b5 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.150281 May 24 10:05:49.561693 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 8/14] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:49 2022] POST /compute/v2.1/flavors => generated 404 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:05:49.573641 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-64a09fe6-085f-46fe-bca0-20f27beeed48 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:49.675048 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-64a09fe6-085f-46fe-bca0-20f27beeed48 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.105234 May 24 10:05:49.675635 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 7/15] 10.222.0.35 () {64 vars in 1335 bytes} [Tue May 24 10:05:49 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:05:51.956527 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-d2b27859-0d30-40e3-9e3a-e0f84339c3b5 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:51.957578 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-d2b27859-0d30-40e3-9e3a-e0f84339c3b5 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001441 May 24 10:05:51.957977 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 9/16] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:52.016491 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-edc125cf-74eb-4ecd-8ee8-73309989663d admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "id": "d4", "name": "ds4G", "disk": 20, "ram": 4096, "rxtx_factor": 1.0}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:52.093913 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-edc125cf-74eb-4ecd-8ee8-73309989663d admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.130250 May 24 10:05:52.094468 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 8/17] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:51 2022] POST /compute/v2.1/flavors => generated 404 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:05:52.106973 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-b9dff181-f1a8-4931-be9d-b09e7790d386 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:52.198368 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-b9dff181-f1a8-4931-be9d-b09e7790d386 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.095111 May 24 10:05:52.199103 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 10/18] 10.222.0.35 () {64 vars in 1335 bytes} [Tue May 24 10:05:52 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:05:54.452937 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-edc125cf-74eb-4ecd-8ee8-73309989663d admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:54.454068 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-edc125cf-74eb-4ecd-8ee8-73309989663d admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 May 24 10:05:54.454466 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 9/19] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:54.520590 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-71f8fbce-cb5f-4916-83cb-97cf1b1046ca admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:54.544989 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-71f8fbce-cb5f-4916-83cb-97cf1b1046ca admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.084441 May 24 10:05:54.545673 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 11/20] 10.222.0.35 () {60 vars in 1291 bytes} [Tue May 24 10:05:54 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:05:57.174352 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-edc125cf-74eb-4ecd-8ee8-73309989663d admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:57.175356 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-edc125cf-74eb-4ecd-8ee8-73309989663d admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001440 May 24 10:05:57.175758 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 10/21] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:57.235972 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-1807dad6-723e-495f-91f5-1828f787bfe9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 512, "vcpus": 1, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "name": "m1.tiny", "id": "1"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:57.321605 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-1807dad6-723e-495f-91f5-1828f787bfe9 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.139678 May 24 10:05:57.322325 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 12/22] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:57 2022] POST /compute/v2.1/flavors => generated 402 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:05:57.335425 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-3f2781a9-4cc9-442b-89b9-a49d1c1de6c1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:57.423528 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-3f2781a9-4cc9-442b-89b9-a49d1c1de6c1 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.091781 May 24 10:05:57.424027 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 11/23] 10.222.0.35 () {64 vars in 1332 bytes} [Tue May 24 10:05:57 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:05:59.804924 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-1807dad6-723e-495f-91f5-1828f787bfe9 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:05:59.806303 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-1807dad6-723e-495f-91f5-1828f787bfe9 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002011 May 24 10:05:59.806809 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 13/24] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:05:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:05:59.880881 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-47b52728-72b0-48f8-94df-d2288d6be1a4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "name": "m1.small", "swap": 0, "id": "2", "os-flavor-access:is_public": true, "vcpus": 1, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:05:59.974313 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-47b52728-72b0-48f8-94df-d2288d6be1a4 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.159615 May 24 10:05:59.974879 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 12/25] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:05:59 2022] POST /compute/v2.1/flavors => generated 405 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:05:59.992523 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-10f90e4e-1e37-400e-91dd-9cfc0f214260 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:00.094467 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-10f90e4e-1e37-400e-91dd-9cfc0f214260 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.107806 May 24 10:06:00.097339 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 14/26] 10.222.0.35 () {64 vars in 1332 bytes} [Tue May 24 10:05:59 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:06:02.601748 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-47b52728-72b0-48f8-94df-d2288d6be1a4 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:02.602759 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-47b52728-72b0-48f8-94df-d2288d6be1a4 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001475 May 24 10:06:02.603300 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 13/27] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:06:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:02.663162 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-0bc58cde-c200-4c0e-b0ed-e49b3a391136 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "3", "swap": 0, "os-flavor-access:is_public": true, "disk": 40, "vcpus": 2, "name": "m1.medium", "ram": 4096}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:02.780147 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-0bc58cde-c200-4c0e-b0ed-e49b3a391136 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.170564 May 24 10:06:02.780708 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 15/28] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:06:02 2022] POST /compute/v2.1/flavors => generated 406 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:06:02.792640 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-4c8727db-d2d9-422e-827e-7006cdb4db67 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:02.915011 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-4c8727db-d2d9-422e-827e-7006cdb4db67 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.125993 May 24 10:06:02.915591 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 14/29] 10.222.0.35 () {64 vars in 1332 bytes} [Tue May 24 10:06:02 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:06:05.183963 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-0bc58cde-c200-4c0e-b0ed-e49b3a391136 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:05.185213 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-0bc58cde-c200-4c0e-b0ed-e49b3a391136 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001796 May 24 10:06:05.186094 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 16/30] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:06:05 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:05.251888 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-5c36b43b-60ca-4cc9-bef0-46d153f9c276 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 80, "ram": 8192, "name": "m1.large", "swap": 0, "id": "4"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:05.357520 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-5c36b43b-60ca-4cc9-bef0-46d153f9c276 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.165777 May 24 10:06:05.358103 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 15/31] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:06:05 2022] POST /compute/v2.1/flavors => generated 405 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:06:05.370360 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-9f17d4ee-8eba-4473-b829-8d6122042f93 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:05.456885 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-9f17d4ee-8eba-4473-b829-8d6122042f93 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090369 May 24 10:06:05.457569 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 17/32] 10.222.0.35 () {64 vars in 1332 bytes} [Tue May 24 10:06:05 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:06:07.836218 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-5c36b43b-60ca-4cc9-bef0-46d153f9c276 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:07.837234 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-5c36b43b-60ca-4cc9-bef0-46d153f9c276 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001461 May 24 10:06:07.837728 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 16/33] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:06:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:07.903019 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-67c8daca-2c94-4022-a416-9a791d946473 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.xlarge", "vcpus": 8, "ram": 16384, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "5", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:07.995414 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-67c8daca-2c94-4022-a416-9a791d946473 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.151282 May 24 10:06:08.002930 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 18/34] 10.222.0.35 () {64 vars in 1282 bytes} [Tue May 24 10:06:07 2022] POST /compute/v2.1/flavors => generated 408 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:06:08.007886 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-8d99d4ff-7346-4fa7-bd42-70e092a4b029 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:08.109714 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-8d99d4ff-7346-4fa7-bd42-70e092a4b029 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.105346 May 24 10:06:08.110303 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 17/35] 10.222.0.35 () {64 vars in 1332 bytes} [Tue May 24 10:06:08 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:06:32.285069 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-67c8daca-2c94-4022-a416-9a791d946473 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:32.286047 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-67c8daca-2c94-4022-a416-9a791d946473 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 May 24 10:06:32.286439 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 19/36] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:06:32 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:32.297732 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:32.346660 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-e9fd14f7-db69-4712-8c64-ba88f1438d85 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:32.362462 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-e9fd14f7-db69-4712-8c64-ba88f1438d85 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.070249 May 24 10:06:32.362995 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 18/37] 10.222.0.35 () {60 vars in 1291 bytes} [Tue May 24 10:06:32 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:06:33.346554 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-67c8daca-2c94-4022-a416-9a791d946473 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:33.347552 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-67c8daca-2c94-4022-a416-9a791d946473 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001407 May 24 10:06:33.347946 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 20/38] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:33.553133 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-07df44e1-f51e-49a9-9c86-068cd09e94aa admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:33.559883 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.wsgi [None req-07df44e1-f51e-49a9-9c86-068cd09e94aa admin admin] HTTP exception thrown: Flavor 42 could not be found. May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-07df44e1-f51e-49a9-9c86-068cd09e94aa admin admin] Request method failure captured: May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Accept: application/json May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Accept-Encoding: gzip, deflate May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Connection: keep-alive May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Content-Length: 0 May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Host: 10.222.0.35 May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Openstack-Api-Version: compute 2.90 May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Openstack-System-Scope: None May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: User-Agent: python-novaclient May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Auth-Token: gAAAAABijK4ptP-ofKlh-_cUI8L2fmoZFnk9AKknHiDhXGiSaXiEs3Y-UX0Qk3oJng9gn3aLIP7baIhKryKx-4R_Du6Ddnc78gFu-HtPx7nzU92FeC9JOJMVd4fgXk-XFN41RCBnZtxEqc8CnVfjHxscYe2vwrhFoUHkBZi_ZznInGHEF1xCp8U May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Domain-Id: None May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Domain-Name: None May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Identity-Status: Confirmed May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Is-Admin-Project: True May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Openstack-Nova-Api-Version: 2.90 May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Domain-Id: default May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Domain-Name: Default May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Name: admin May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Role: member,admin,reader May 24 10:06:33.563146 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Roles: member,admin,reader May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_d39d97f2faca4faba12c96da090b5017"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant: admin May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant-Name: admin May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User: admin May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Domain-Id: default May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Domain-Name: Default May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Id: db8edbaf5787451c9cd851df007d0f35 May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Name: admin May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: method: > May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: exception: Flavor 42 could not be found. May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: action_args: {'id': '42'} May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:06:33.564985 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:33.566097 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:33.567475 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-07df44e1-f51e-49a9-9c86-068cd09e94aa admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103800) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:06:33.568842 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-07df44e1-f51e-49a9-9c86-068cd09e94aa admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.063748 May 24 10:06:33.568842 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 19/39] 10.222.0.35 () {60 vars in 1190 bytes} [Tue May 24 10:06:33 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 24 10:06:33.575057 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-9c67eb46-af20-4536-8f1f-24e429a07319 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:33.578080 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:33.597219 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-9c67eb46-af20-4536-8f1f-24e429a07319 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026353 May 24 10:06:33.597781 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 21/40] 10.222.0.35 () {60 vars in 1210 bytes} [Tue May 24 10:06:33 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:06:33.608679 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-ceec3d0d-015d-44e5-b45d-4ca859fa39b8 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:33.629802 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-ceec3d0d-015d-44e5-b45d-4ca859fa39b8 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025901 May 24 10:06:33.630345 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 20/41] 10.222.0.35 () {60 vars in 1210 bytes} [Tue May 24 10:06:33 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:06:33.640140 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-bac0ae79-2550-4c35-8925-cd96f1ffe340 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:33.647247 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.wsgi [None req-bac0ae79-2550-4c35-8925-cd96f1ffe340 admin admin] HTTP exception thrown: Flavor 42 could not be found. May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-bac0ae79-2550-4c35-8925-cd96f1ffe340 admin admin] Request method failure captured: May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Accept: application/json May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Accept-Encoding: gzip, deflate May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Connection: keep-alive May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Content-Length: 0 May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Host: 10.222.0.35 May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Openstack-Api-Version: compute 2.90 May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Openstack-System-Scope: None May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: User-Agent: python-novaclient May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Auth-Token: gAAAAABijK4ptP-ofKlh-_cUI8L2fmoZFnk9AKknHiDhXGiSaXiEs3Y-UX0Qk3oJng9gn3aLIP7baIhKryKx-4R_Du6Ddnc78gFu-HtPx7nzU92FeC9JOJMVd4fgXk-XFN41RCBnZtxEqc8CnVfjHxscYe2vwrhFoUHkBZi_ZznInGHEF1xCp8U May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Domain-Id: None May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Domain-Name: None May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Identity-Status: Confirmed May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Is-Admin-Project: True May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Openstack-Nova-Api-Version: 2.90 May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Domain-Id: default May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Domain-Name: Default May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Name: admin May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Role: member,admin,reader May 24 10:06:33.650596 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Roles: member,admin,reader May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_d39d97f2faca4faba12c96da090b5017"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant: admin May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant-Name: admin May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User: admin May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Domain-Id: default May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Domain-Name: Default May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Id: db8edbaf5787451c9cd851df007d0f35 May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Name: admin May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: method: > May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: exception: Flavor 42 could not be found. May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: action_args: {'id': '42'} May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:06:33.652785 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:33.653836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:33.655097 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-bac0ae79-2550-4c35-8925-cd96f1ffe340 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103799) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:06:33.656357 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-bac0ae79-2550-4c35-8925-cd96f1ffe340 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015855 May 24 10:06:33.656357 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 22/42] 10.222.0.35 () {60 vars in 1190 bytes} [Tue May 24 10:06:33 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 18 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 24 10:06:34.514982 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-ceec3d0d-015d-44e5-b45d-4ca859fa39b8 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:34.515990 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-ceec3d0d-015d-44e5-b45d-4ca859fa39b8 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001434 May 24 10:06:34.516395 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 21/43] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:34.707464 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-c51e7ae9-e887-40e9-8fa1-29ef2f84abfa admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:34.713048 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.wsgi [None req-c51e7ae9-e887-40e9-8fa1-29ef2f84abfa admin admin] HTTP exception thrown: Flavor 84 could not be found. May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-c51e7ae9-e887-40e9-8fa1-29ef2f84abfa admin admin] Request method failure captured: May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Accept: application/json May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Accept-Encoding: gzip, deflate May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Connection: keep-alive May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Content-Length: 0 May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Host: 10.222.0.35 May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Openstack-Api-Version: compute 2.90 May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Openstack-System-Scope: None May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: User-Agent: python-novaclient May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Auth-Token: gAAAAABijK4qqM4UyV9aM1T_d3pm981y7Ngu_lpBCj16Uby1dd-iW3NsGdzANQ01lNOJ0__FvaLFWWbtjIEuCTdSiP0cImVGkOjx8-UShBiN1smSxYok7v7R5W-B8h7wsPMJY3t3J9I30cJw-0Oqbjwh-DAHKnQHntLFTNO3RHE6mxJvUkmfAyA May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Domain-Id: None May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Domain-Name: None May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Identity-Status: Confirmed May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Is-Admin-Project: True May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Openstack-Nova-Api-Version: 2.90 May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Domain-Id: default May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Domain-Name: Default May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Name: admin May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Role: member,admin,reader May 24 10:06:34.714848 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Roles: member,admin,reader May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_d39d97f2faca4faba12c96da090b5017"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant: admin May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant-Name: admin May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User: admin May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Domain-Id: default May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Domain-Name: Default May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Id: db8edbaf5787451c9cd851df007d0f35 May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Name: admin May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: method: > May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: exception: Flavor 84 could not be found. May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: action_args: {'id': '84'} May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:06:34.716505 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:34.717522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:34.718940 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-c51e7ae9-e887-40e9-8fa1-29ef2f84abfa admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103799) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:06:34.720214 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-c51e7ae9-e887-40e9-8fa1-29ef2f84abfa admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.063976 May 24 10:06:34.720214 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 23/44] 10.222.0.35 () {60 vars in 1190 bytes} [Tue May 24 10:06:34 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 24 10:06:34.726046 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-7cebc942-c090-409e-9083-3952db4a50d3 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:34.741980 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-7cebc942-c090-409e-9083-3952db4a50d3 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019567 May 24 10:06:34.742508 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 22/45] 10.222.0.35 () {60 vars in 1210 bytes} [Tue May 24 10:06:34 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:06:34.752874 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-e1906bbf-57e2-4149-a428-047dcfc41a1b admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:34.767621 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-e1906bbf-57e2-4149-a428-047dcfc41a1b admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018571 May 24 10:06:34.768131 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 24/46] 10.222.0.35 () {60 vars in 1210 bytes} [Tue May 24 10:06:34 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:06:34.777538 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-ed511576-74c6-4c5c-9332-1b868cf97c00 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:34.782994 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.wsgi [None req-ed511576-74c6-4c5c-9332-1b868cf97c00 admin admin] HTTP exception thrown: Flavor 84 could not be found. May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-ed511576-74c6-4c5c-9332-1b868cf97c00 admin admin] Request method failure captured: May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Accept: application/json May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Accept-Encoding: gzip, deflate May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Connection: keep-alive May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Content-Length: 0 May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Host: 10.222.0.35 May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Openstack-Api-Version: compute 2.90 May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Openstack-System-Scope: None May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: User-Agent: python-novaclient May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Auth-Token: gAAAAABijK4qqM4UyV9aM1T_d3pm981y7Ngu_lpBCj16Uby1dd-iW3NsGdzANQ01lNOJ0__FvaLFWWbtjIEuCTdSiP0cImVGkOjx8-UShBiN1smSxYok7v7R5W-B8h7wsPMJY3t3J9I30cJw-0Oqbjwh-DAHKnQHntLFTNO3RHE6mxJvUkmfAyA May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Domain-Id: None May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Domain-Name: None May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Identity-Status: Confirmed May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Is-Admin-Project: True May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Openstack-Nova-Api-Version: 2.90 May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Domain-Id: default May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Domain-Name: Default May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Name: admin May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Role: member,admin,reader May 24 10:06:34.784741 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Roles: member,admin,reader May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_d39d97f2faca4faba12c96da090b5017"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant: admin May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant-Name: admin May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User: admin May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Domain-Id: default May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Domain-Name: Default May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Id: db8edbaf5787451c9cd851df007d0f35 May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Name: admin May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: method: > May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: exception: Flavor 84 could not be found. May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: action_args: {'id': '84'} May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:06:34.786131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:34.786958 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:34.788106 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-ed511576-74c6-4c5c-9332-1b868cf97c00 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103800) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:06:34.789187 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-ed511576-74c6-4c5c-9332-1b868cf97c00 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012286 May 24 10:06:34.789187 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 23/47] 10.222.0.35 () {60 vars in 1190 bytes} [Tue May 24 10:06:34 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 24 10:06:35.643708 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-e1906bbf-57e2-4149-a428-047dcfc41a1b admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:35.644697 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-e1906bbf-57e2-4149-a428-047dcfc41a1b admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001415 May 24 10:06:35.645085 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 25/48] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:35.834844 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-1d01e04e-78bc-453d-9f86-d0dd56a25bdf admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:35.841471 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.wsgi [None req-1d01e04e-78bc-453d-9f86-d0dd56a25bdf admin admin] HTTP exception thrown: Flavor 451 could not be found. May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-1d01e04e-78bc-453d-9f86-d0dd56a25bdf admin admin] Request method failure captured: May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Accept: application/json May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Accept-Encoding: gzip, deflate May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Connection: keep-alive May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Content-Length: 0 May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Host: 10.222.0.35 May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Openstack-Api-Version: compute 2.90 May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Openstack-System-Scope: None May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: User-Agent: python-novaclient May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Auth-Token: gAAAAABijK4rt63Ns3HGowy9RUlfJ9QEQ_7duQ88Kkj7u8W2fb3KuC-x3xSo-dLhPG4qUuKbMLVuKl8-15KFklaeRlFdCbKWFy8GsfH_jCLD4aSDTOr7WUbo9irSC5AKzKZU1FL764tpKg_nGJIc372v5g2ypSashbvodmr_D3a7UeTw2iV3fRA May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Domain-Id: None May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Domain-Name: None May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Identity-Status: Confirmed May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Is-Admin-Project: True May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Openstack-Nova-Api-Version: 2.90 May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Domain-Id: default May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Domain-Name: Default May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Project-Name: admin May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Role: member,admin,reader May 24 10:06:35.842978 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Roles: member,admin,reader May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_d39d97f2faca4faba12c96da090b5017"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant: admin May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-Tenant-Name: admin May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User: admin May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Domain-Id: default May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Domain-Name: Default May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Id: db8edbaf5787451c9cd851df007d0f35 May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: X-User-Name: admin May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: method: > May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: exception: Flavor 451 could not be found. May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: action_args: {'id': '451'} May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:06:35.844488 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:35.845474 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: ERROR nova.api.openstack.wsgi May 24 10:06:35.846757 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-1d01e04e-78bc-453d-9f86-d0dd56a25bdf admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103800) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:06:35.848016 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-1d01e04e-78bc-453d-9f86-d0dd56a25bdf admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.061420 May 24 10:06:35.848016 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 24/49] 10.222.0.35 () {60 vars in 1193 bytes} [Tue May 24 10:06:35 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 24 10:06:35.854357 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-8d25d14f-f89b-4f76-9a19-9408d11d235c admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:35.868412 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-8d25d14f-f89b-4f76-9a19-9408d11d235c admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017626 May 24 10:06:35.868957 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 26/50] 10.222.0.35 () {60 vars in 1210 bytes} [Tue May 24 10:06:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:06:35.878464 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:35.891929 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.016946 May 24 10:06:35.892454 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 25/51] 10.222.0.35 () {60 vars in 1210 bytes} [Tue May 24 10:06:35 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:06:35.901319 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-d9bd80ba-e9b2-4c17-aa90-5089dba2eebe admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:35.907149 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.wsgi [None req-d9bd80ba-e9b2-4c17-aa90-5089dba2eebe admin admin] HTTP exception thrown: Flavor 451 could not be found. May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-d9bd80ba-e9b2-4c17-aa90-5089dba2eebe admin admin] Request method failure captured: May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Accept: application/json May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Accept-Encoding: gzip, deflate May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Connection: keep-alive May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Content-Length: 0 May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Host: 10.222.0.35 May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Openstack-Api-Version: compute 2.90 May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Openstack-System-Scope: None May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: User-Agent: python-novaclient May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Auth-Token: gAAAAABijK4rt63Ns3HGowy9RUlfJ9QEQ_7duQ88Kkj7u8W2fb3KuC-x3xSo-dLhPG4qUuKbMLVuKl8-15KFklaeRlFdCbKWFy8GsfH_jCLD4aSDTOr7WUbo9irSC5AKzKZU1FL764tpKg_nGJIc372v5g2ypSashbvodmr_D3a7UeTw2iV3fRA May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Domain-Id: None May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Domain-Name: None May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Identity-Status: Confirmed May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Is-Admin-Project: True May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Openstack-Nova-Api-Version: 2.90 May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Domain-Id: default May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Domain-Name: Default May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Project-Name: admin May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Role: member,admin,reader May 24 10:06:35.908890 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Roles: member,admin,reader May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_d39d97f2faca4faba12c96da090b5017"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/d39d97f2faca4faba12c96da090b5017"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant: admin May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant-Id: d39d97f2faca4faba12c96da090b5017 May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-Tenant-Name: admin May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User: admin May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Domain-Id: default May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Domain-Name: Default May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Id: db8edbaf5787451c9cd851df007d0f35 May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: X-User-Name: admin May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: method: > May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: exception: Flavor 451 could not be found. May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: action_args: {'id': '451'} May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:06:35.910522 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:35.911553 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: ERROR nova.api.openstack.wsgi May 24 10:06:35.912822 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-d9bd80ba-e9b2-4c17-aa90-5089dba2eebe admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103799) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:06:35.914205 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-d9bd80ba-e9b2-4c17-aa90-5089dba2eebe admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012426 May 24 10:06:35.914205 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 27/52] 10.222.0.35 () {60 vars in 1193 bytes} [Tue May 24 10:06:35 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) May 24 10:06:36.767813 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:36.768819 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 May 24 10:06:36.772395 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 26/53] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:36 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:36.959334 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-feb6fec4-126a-4975-9cb3-3ef31046ec7f admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:37.031579 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-feb6fec4-126a-4975-9cb3-3ef31046ec7f admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.125393 May 24 10:06:37.032054 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 28/54] 10.222.0.35 () {64 vars in 1235 bytes} [Tue May 24 10:06:36 2022] POST /compute/v2.1/flavors => generated 404 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:06:37.927305 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:37.928341 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001454 May 24 10:06:37.928738 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 27/55] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:38.115960 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-5ed0afc9-2272-47c5-9969-04027eec39f0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:38.195679 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-5ed0afc9-2272-47c5-9969-04027eec39f0 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.130753 May 24 10:06:38.196225 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 29/56] 10.222.0.35 () {64 vars in 1235 bytes} [Tue May 24 10:06:38 2022] POST /compute/v2.1/flavors => generated 405 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:06:39.069419 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:39.070422 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001435 May 24 10:06:39.070815 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 28/57] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:39.260729 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-c142d28d-9e49-4e05-a802-9be815bb9548 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:39.348860 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-c142d28d-9e49-4e05-a802-9be815bb9548 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.140213 May 24 10:06:39.349344 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 30/58] 10.222.0.35 () {64 vars in 1235 bytes} [Tue May 24 10:06:39 2022] POST /compute/v2.1/flavors => generated 407 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:06:40.213177 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:40.214436 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001439 May 24 10:06:40.215765 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 29/59] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:40.405061 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-c7745c5f-2202-4417-8721-e46ba0d90d08 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:06:40.486256 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-c7745c5f-2202-4417-8721-e46ba0d90d08 admin admin] 10.222.0.35 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.132976 May 24 10:06:40.486836 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 31/60] 10.222.0.35 () {64 vars in 1235 bytes} [Tue May 24 10:06:40 2022] POST /compute/v2.1/flavors => generated 409 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:06:41.352673 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:41.353733 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001515 May 24 10:06:41.354131 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 30/61] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:06:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:06:41.546136 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-11223c46-c89d-462d-9ad7-3dc6e99ba0ac admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:06:41.565676 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-11223c46-c89d-462d-9ad7-3dc6e99ba0ac admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.073985 May 24 10:06:41.566234 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 32/62] 10.222.0.35 () {60 vars in 1202 bytes} [Tue May 24 10:06:41 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:07:43.499244 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:07:43.500243 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-6aff8b9b-ee80-422d-bc86-7c58922f47db admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001423 May 24 10:07:43.500665 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 31/63] 10.222.0.35 () {54 vars in 930 bytes} [Tue May 24 10:07:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:07:43.507870 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:07:43.561442 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-8f7c287e-12be-4536-974e-bd4fa3ed67e1 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:07:43.581254 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-8f7c287e-12be-4536-974e-bd4fa3ed67e1 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.074767 May 24 10:07:43.581806 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 33/64] 10.222.0.35 () {60 vars in 1291 bytes} [Tue May 24 10:07:43 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:07:43.602458 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG dbcounter [-] [103800] Writing DB stats nova_api:SELECT=14 {{(pid=103800) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:07:43.607040 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-fe3a35db-c0c8-4941-bb25-a1b1b626783d admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:07:43.614569 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-fe3a35db-c0c8-4941-bb25-a1b1b626783d admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013524 May 24 10:07:43.615081 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 32/65] 10.222.0.35 () {60 vars in 1282 bytes} [Tue May 24 10:07:43 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:07:43.627107 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-a5917a6d-2c2a-464a-9ae9-e23f3aaca1d4 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:07:43.634769 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-a5917a6d-2c2a-464a-9ae9-e23f3aaca1d4 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011251 May 24 10:07:43.635273 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 34/66] 10.222.0.35 () {60 vars in 1285 bytes} [Tue May 24 10:07:43 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:07:43.646775 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:07:43.655083 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012166 May 24 10:07:43.655599 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 33/67] 10.222.0.35 () {60 vars in 1285 bytes} [Tue May 24 10:07:43 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:07:43.669167 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-363b9d0e-2c33-4686-bc85-64d48d6e0093 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:07:43.676851 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-363b9d0e-2c33-4686-bc85-64d48d6e0093 admin admin] 10.222.0.35 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012006 May 24 10:07:43.677396 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 35/68] 10.222.0.35 () {60 vars in 1282 bytes} [Tue May 24 10:07:43 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:11:22.855238 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:22.856257 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002844 May 24 10:11:22.856660 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 34/69] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:11:22 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:11:23.045849 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_api:SELECT=6 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:23.160684 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-09e6eb35-b937-4c66-8434-26e56420be81 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:23.161915 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:11:23.161915 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: warnings.warn(msg) May 24 10:11:23.163135 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-09e6eb35-b937-4c66-8434-26e56420be81 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:23.163718 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-09e6eb35-b937-4c66-8434-26e56420be81 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:23.164520 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-09e6eb35-b937-4c66-8434-26e56420be81 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:23.164952 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-09e6eb35-b937-4c66-8434-26e56420be81 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:23.199902 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-09e6eb35-b937-4c66-8434-26e56420be81 admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.155513 May 24 10:11:23.200507 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 36/70] 10.222.0.35 () {60 vars in 1193 bytes} [Tue May 24 10:11:23 2022] GET /compute/v2.1/os-services => generated 791 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:11:24.092847 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:24.093906 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 May 24 10:11:24.094298 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 35/71] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:11:24 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:11:24.288661 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-85e8373e-f7d3-44c2-b233-3a920dcf3f51 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:24.289878 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:11:24.289878 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: warnings.warn(msg) May 24 10:11:24.291068 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-85e8373e-f7d3-44c2-b233-3a920dcf3f51 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:24.291518 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-85e8373e-f7d3-44c2-b233-3a920dcf3f51 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:24.292339 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-85e8373e-f7d3-44c2-b233-3a920dcf3f51 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:24.292773 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-85e8373e-f7d3-44c2-b233-3a920dcf3f51 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:24.312545 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-85e8373e-f7d3-44c2-b233-3a920dcf3f51 admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.073798 May 24 10:11:24.313051 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 37/72] 10.222.0.35 () {60 vars in 1193 bytes} [Tue May 24 10:11:24 2022] GET /compute/v2.1/os-services => generated 791 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:11:35.583498 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:35.584505 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 May 24 10:11:35.584893 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 36/73] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:11:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:11:35.729525 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_cell0:SELECT=4 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:35.730252 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_cell1:SELECT=4 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:35.730883 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG dbcounter [-] [103799] Writing DB stats nova_api:SELECT=8 {{(pid=103799) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:35.778708 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-0e791d57-886f-45f4-9ed6-4d2efb420d22 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:35.779397 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:11:35.779397 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: warnings.warn(msg) May 24 10:11:35.780577 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-0e791d57-886f-45f4-9ed6-4d2efb420d22 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:35.781019 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-0e791d57-886f-45f4-9ed6-4d2efb420d22 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:35.781841 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-0e791d57-886f-45f4-9ed6-4d2efb420d22 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:35.782300 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-0e791d57-886f-45f4-9ed6-4d2efb420d22 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:35.813084 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-0e791d57-886f-45f4-9ed6-4d2efb420d22 admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.083536 May 24 10:11:35.813084 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 38/74] 10.222.0.35 () {60 vars in 1193 bytes} [Tue May 24 10:11:35 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:11:36.701451 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: DEBUG nova.api.openstack.wsgi [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] Calling method '>' {{(pid=103800) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:36.702458 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: INFO nova.api.openstack.requestlog [None req-b5755248-01f7-4d31-a272-0c78a6c796b3 admin admin] 10.222.0.35 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001456 May 24 10:11:36.702853 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: [pid: 103800|app: 0|req: 37/75] 10.222.0.35 () {56 vars in 1074 bytes} [Tue May 24 10:11:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:11:36.892439 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG nova.api.openstack.wsgi [None req-820738a4-dc75-41e2-889e-882951072b52 admin admin] Calling method '>' {{(pid=103799) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:11:36.894263 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:11:36.894263 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: warnings.warn(msg) May 24 10:11:36.895454 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-820738a4-dc75-41e2-889e-882951072b52 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:36.895896 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-820738a4-dc75-41e2-889e-882951072b52 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:36.896674 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-820738a4-dc75-41e2-889e-882951072b52 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:11:36.897101 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: DEBUG oslo_concurrency.lockutils [None req-820738a4-dc75-41e2-889e-882951072b52 admin admin] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103799) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:11:36.919340 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: INFO nova.api.openstack.requestlog [None req-820738a4-dc75-41e2-889e-882951072b52 admin admin] 10.222.0.35 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.077901 May 24 10:11:36.919815 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: [pid: 103799|app: 0|req: 39/76] 10.222.0.35 () {60 vars in 1193 bytes} [Tue May 24 10:11:36 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:12:36.314849 dv-cinder-843118-2-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... May 24 10:12:36.315416 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: Tue May 24 10:12:36 2022 - graceful shutdown triggered... May 24 10:12:36.315873 dv-cinder-843118-2-iscsi devstack@n-api.service[103800]: Gracefully killing worker 2 (pid: 103800)... May 24 10:12:36.316289 dv-cinder-843118-2-iscsi devstack@n-api.service[103799]: Gracefully killing worker 1 (pid: 103799)... May 24 10:12:37.319860 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: worker 1 buried after 1 seconds May 24 10:12:37.319860 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: worker 2 buried after 1 seconds May 24 10:12:37.320404 dv-cinder-843118-2-iscsi devstack@n-api.service[103798]: goodbye to uWSGI. May 24 10:12:37.321435 dv-cinder-843118-2-iscsi systemd[1]: devstack@n-api.service: Succeeded. May 24 10:12:37.322002 dv-cinder-843118-2-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. May 24 10:12:37.324765 dv-cinder-843118-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... May 24 10:12:37.341538 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 10:12:37 2022] *** May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: nodename: dv-cinder-843118-2-iscsi May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: machine: x86_64 May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: clock source: unix May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: pcre jit disabled May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: detected number of CPU cores: 4 May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: current working directory: / May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: detected binary path: /usr/bin/uwsgi-core May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: your processes number limit is 47680 May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: your memory page size is 4096 bytes May 24 10:12:37.362723 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: detected max file descriptor number: 2048 May 24 10:12:37.363607 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: lock engine: pthread robust mutexes May 24 10:12:37.363607 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: thunder lock: enabled May 24 10:12:37.363607 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 May 24 10:12:37.363607 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 10:12:37.412034 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: Python main interpreter initialized at 0x56067b0979c0 May 24 10:12:37.412034 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: python threads support enabled May 24 10:12:37.412034 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: your server socket listen backlog is limited to 100 connections May 24 10:12:37.412034 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: your mercy for graceful operations on workers is 90 seconds May 24 10:12:37.412578 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: mapped 403077 bytes (393 KB) for 2 cores May 24 10:12:37.412578 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: *** Operational MODE: preforking *** May 24 10:12:37.412804 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: *** uWSGI is running in multiple interpreter mode *** May 24 10:12:37.412804 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: spawned uWSGI master process (pid: 118217) May 24 10:12:37.412834 dv-cinder-843118-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. May 24 10:12:37.413461 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: spawned uWSGI worker 1 (pid: 118232, cores: 1) May 24 10:12:37.413809 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: spawned uWSGI worker 2 (pid: 118233, cores: 1) May 24 10:12:37.413964 dv-cinder-843118-2-iscsi devstack@n-api.service[118217]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 10:12:38.616637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 24 10:12:38.621610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. May 24 10:12:39.428145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:12:39.428145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: from cryptography.utils import int_from_bytes May 24 10:12:39.428421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:12:39.428421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: from cryptography.utils import int_from_bytes May 24 10:12:39.428569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:12:39.428569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: from cryptography.utils import int_from_bytes May 24 10:12:39.428944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 10:12:39.428944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: from cryptography.utils import int_from_bytes May 24 10:12:41.930053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 10:12:41.930607 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 10:12:41.930917 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 10:12:41.931128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 10:12:41.931384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 10:12:41.931779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.932113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.932445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.932770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.933103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.933449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.933777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.934101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.934435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.934765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.935096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.935425 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843118-2-iscsi {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.935754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.936065 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.936377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.936699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.937003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.937358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.937669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.938177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.939572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.939572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.939572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.939843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.940171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.940495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.940813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.941452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.941775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.942106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.942514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843118-2-iscsi {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.942840 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.943174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.943483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.943903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.944224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.944545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.944871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.945192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.945550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.945859 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.946182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.946508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.946830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.947137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.947452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.947764 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.948085 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.948393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.948702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.948955 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.949301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.949617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.949942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.950269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.950524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.950789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.951104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.951424 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.951737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.952049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.952366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.952677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.952990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.953322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.953636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.953946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.954272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.954595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.954927 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.955242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.958701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.960763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.961059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.961236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.961416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.961736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.962054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.962375 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.962691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.962997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.963309 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.963625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.963940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.964252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.964563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.964927 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.965192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.965588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.965897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.966221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.966547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.966861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.967173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.967485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.967797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.968122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.968435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.968747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.969076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.969410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.969720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.970044 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.970361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.970674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.970997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.971301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.971472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 10:12:41.972000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.972179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 10:12:41.972919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.973496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 10:12:41.973687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.973687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.973987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 10:12:41.974382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.974551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 10:12:41.974701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.974701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.975023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.975238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.975406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.975578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.975807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.975968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.976122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.976277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.976438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.976594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.976752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.976913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.977063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.977227 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.977427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.978618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978710 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.978791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-843118-2-iscsi {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.978870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.978950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.979022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.979091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.979167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.979239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.979315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.979398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.979479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.979555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.979772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.979892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.980178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.980349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.980689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.980773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.981007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.981103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.981475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.981634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.981781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.981926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.982071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.982217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.982379 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.982536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.982690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.982834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.982990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.983164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.983704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.983863 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.984018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.984175 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.984325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.984571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.984571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.984853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-843118-2-iscsi {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.985021 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.985176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.985435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.985590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.985745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.985984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.986133 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.986279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.986443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.986591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.986736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.986889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.987057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.987219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.987377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.987525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.987671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.987826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.987981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.988152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.988308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.988504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.988734 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.988958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.989167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.989167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.989599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.989756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.989912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.990066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.990225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.990381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.990581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.990749 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.990913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.991068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.991224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.991371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.991535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.991684 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.991880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.992033 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.992190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.992344 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.992491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.992636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.992791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.992937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.993093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.993255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.993815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.993992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.994155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.994320 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.994467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.994743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.994905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.995062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.995218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.995376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.995618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.995769 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.995929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.996167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.996327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.996489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.996636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.996788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.996943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.997089 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.997249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.997453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.997610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.997765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.997923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.998081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.998237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.998383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.998529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.998683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.998830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.998983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.999203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.999361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.999516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.35 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.999671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:41.999825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.35 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:41.999971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.000139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.000293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.000439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.000597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.000752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.000911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.001065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.001221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.001429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.001586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.001733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.001888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.002053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.002201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.002346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.002491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.002636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.002790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.002937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.003084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.003231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.003385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.003553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.004160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.004331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.004501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.004676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.004903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.005056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.005210 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.005400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.005549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.005777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.005928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.006084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.006326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.006326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.006635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.006802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.006970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.007462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.007633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.007782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.007928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.008081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.008229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.008374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.008530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.008686 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.008840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.008987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.009144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.009314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.009490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.009638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.009792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.009938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.010092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.010237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.010392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.010549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.010711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.010861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.011017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.011174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.011321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.011466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.011613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.011811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.011972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.012127 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.012282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.012441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.012603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.012750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.012904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.013071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.013226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.013431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.013590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.013738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.014271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.014437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.014585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.014741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.014918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.015264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.015424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.015582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.015766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.015961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.016121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.016268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.016513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.016672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.016828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.016990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.017146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.017454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.017624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.017785 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.017942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.018109 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.018258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.018404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.018549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.018702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.018849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.019012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.019669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.019820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.019976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.020132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.020290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.020448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.020613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.020773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.020920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.020920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.021214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.021395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.021543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.021689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.021840 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.022036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 10:12:42.022036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.022036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.022478 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.022774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.022934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.023089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.023246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.023403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.023551 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.023551 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.023835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.024006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.024526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.024681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.024837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.024996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.025150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.025528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.025683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.025839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.025994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.026141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.026298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.026462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.026704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.026863 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027175 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.027935 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.028081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.028226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.028450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.028607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.028762 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.028909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.029062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.029231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.029402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.029550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.029708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.029864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.030018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.030175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.030322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.030470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.031302 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.031465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.031625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.031772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.031918 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.032073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.032229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.032419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.032568 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.032723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.032878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.033833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.034131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.034723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.035594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.035763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.035920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.036083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.036239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.036398 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.036545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.036719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.036976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.037128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.037364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.037519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.037679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.037840 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.037996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.038143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.038297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.038453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.038606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.038753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.038910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.039207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.039378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.039525 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.039680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.039835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.039998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.040146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.040300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.040446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.040600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.040765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.040927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.041082 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.041239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.041408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.041408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.041699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.041855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.042944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.043112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.043269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.043424 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.043582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.043737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.043884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.044029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.044176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.044985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.045227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.045433 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.046145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.046306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.046476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.046641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.046798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.047950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.048096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.048241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.048388 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.048556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.048703 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.048857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.049988 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.050144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.050330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.050503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.050668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.050827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.050985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.051139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.051286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.051434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.051580 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.051726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.051881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.052817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.053094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.053247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.053417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.053611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.053768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.053925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.054073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.054226 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.054380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.054551 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.055087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.055260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.055410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.055556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.055714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.055899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.056064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.056338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.056490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.056636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.056793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.056942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.057107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.057254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.057254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.057587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.057751 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.057905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.058066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.058231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.058391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.058552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.058719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.058874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.059036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.059197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.059357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.059595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.059750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.059911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.060064 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.060224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.060376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.060542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.060713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.060879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.061032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.061032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.061480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.061651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.061806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.061967 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.062129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.062291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.062443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.062609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.062771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.062931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.063092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.063252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.063424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.063592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.063745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.063905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.064059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.064218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.064381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.064533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.064686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.064951 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.065435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.065589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.065755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.065955 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.066127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.066416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.066579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.066743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.067925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.068099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.068253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.068413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.068566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.068735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.068907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.069071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.069224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.069506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.069670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.069838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.069998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.070161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.070499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.070698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.070889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.071090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.071281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.071471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.071660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.071849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.072048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.073599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.073777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.073943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.074097 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.074266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.074436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.074600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.074763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.074942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.075471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.075644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.075753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.075881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.075958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076764 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.076934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.077879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.078973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.079919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.080919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.081004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.081095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.081095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.081276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.081365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.081463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082476 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.082915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.083926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.084990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085082 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085776 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.085952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.086983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.087946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088759 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.088944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.089941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.090949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.091169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.091273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.091485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.091579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.091817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.091911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.092122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.092245 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.092450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.092550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.092763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.092857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.093075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.093295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.093404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.093624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.093726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.093943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.094037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.094279 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.094380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.094609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.094703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.094942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.095960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.096194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.096295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.096523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.096617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.096831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.096923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.097138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.097475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.097573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.097702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.097853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.098077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.098170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.098355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.098457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.098677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.098771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099759 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.099976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.100076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.100282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.100376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.100598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.100699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.100913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.101992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.102181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.102297 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.102484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.102588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.102796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.102907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.103105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.103206 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.103420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.103511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.103725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.103835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104030 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.104968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.105066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.105333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.105431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.105620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.105757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.105938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.106960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.107162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.107254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.107470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.107562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.107779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.107873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.108091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.108199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.108405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.108497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.108710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.108802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.109963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.110995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.111184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.111291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.111485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.111578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.111788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.111890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.112110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.112203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.112419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.112520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.112726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.113347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.113609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.113744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.113884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.113986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.114221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.114412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.114535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.114725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.114846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.115089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.115197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.115405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.115506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.115726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.115857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116461 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.116980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.117090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.117324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.117435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.117651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.117766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.117958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.118056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.118281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.118372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.118596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.118690 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.118904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.119950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.120186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.120287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.120510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.120610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.120833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.120939 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.121141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.121247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.121487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.121593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.121801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.121901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.122111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.122219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.122422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.122515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.122731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.122826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.123046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.123148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.123362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.123455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.123676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.123917 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.124974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.125190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.125314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.125528 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.125621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.125849 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.125981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.126159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.126284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.126465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.126584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.126778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.126899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.127080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.127173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.127394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.127488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.127697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.127805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.128041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.128130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.128383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.129043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.129258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.129603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.129697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.129926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.130021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.130247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.130355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.130580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.130672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.130904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131868 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.131980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.132189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.132295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.132507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.132609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.132824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.132926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.133141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.133236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.133471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.133565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.133784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.133876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.134098 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.134202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.134403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.134495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.134712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.134820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.135018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.135121 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.135344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.135439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.135659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.135766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.136991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.137172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.137432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.137541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.137747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.137851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.138059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.138349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.138487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.138793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.139136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.139236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.139467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.139559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.139777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.139904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.140095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.140180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.140409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.140501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.140738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.140822 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.141063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.141147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.141394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.141483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.141718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.141961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.142063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.142289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.142380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.142645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.142962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.143284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.143705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.144051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.144448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.144761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.144857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.145096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.145196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.145430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.145526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.145749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.146067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.146247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.146375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.146485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.146738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.146858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147121 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.147939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.148116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.148250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.148459 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.148554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.148783 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.148887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.149106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.149198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.149462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.149571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.149784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.149895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.150100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.150200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.150417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.150511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.150737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.150844 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.151974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.152076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.152279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.152372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.152586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.152688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.152911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153882 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.153976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.154196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.154298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.154513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.154613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.154823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.154915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.155143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.155247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.155454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.155556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.155816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.155919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.156099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.156243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.156407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.156527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.156713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.156824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.157024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.157140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.157349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.157445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.157655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.157747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.169406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170042 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.170979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171444 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.171878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.172944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173729 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.173929 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.174923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.175022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.175116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.175209 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.175846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.175938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.176983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177859 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.177952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178396 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.178931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.179985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180115 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.180931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181379 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181647 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.181923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.182963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183302 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.183919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.184950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185304 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.185918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.186078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.186164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.186256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.186350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.186550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.186901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.187973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.188194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.188295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.188508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.188601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.188817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.188911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.189141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.189235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.189537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.189634 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.189847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.189937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.190165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.190266 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.190478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.190573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.190784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.190878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.191653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.191853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.191958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.192178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.192383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.192526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.192620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.192834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.192937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.193173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.193350 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.198984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199860 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.199946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200030 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200648 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.200904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201525 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201873 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.201966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202793 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.202981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203170 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203170 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 10:12:42.203339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.203734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.204035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.204342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.204665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.204975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.205303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.205621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.205934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.206244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.206614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.206925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.207276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.207661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.207978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.208290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.208609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.208949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.209291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.209619 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.209942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.210259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.210581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.210894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.211206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.211516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.211826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.212144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.212475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.212788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.213095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.213553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.213871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.214192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.214514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.214824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.215146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.215470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.215784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.216104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.216422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.216753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.217092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.217447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.217924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.218074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.218378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.218683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.218991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.219309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.219615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.219917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.220218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.220526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.220831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.221132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.221469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.221774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.222082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.222428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.222733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.223065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.223371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.223673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.223974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.224284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.224588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.224907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.225194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.225526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.226712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.226712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.226712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.226971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.227067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.227384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.227704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.228016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.228336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.228657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.228971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.229293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.229636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.229950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.230265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.230582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.230890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.231199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.231509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.231842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.232174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.232482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.232809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.233132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.233482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.233804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.234122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.234453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.234770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.235083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.235403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.235718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.236033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.236361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.236683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.237012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.237368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.237683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.238548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.238878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.239226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.239546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.239894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.240224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.240539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.240856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.241166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 10:12:42.241445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=118232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 10:12:42.262023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO dbcounter [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Registered counter for database nova_api May 24 10:12:42.284428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_db.sqlalchemy.engines [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.292403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writer thread running {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.300272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO dbcounter [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Registered counter for database nova_api May 24 10:12:42.320303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_db.sqlalchemy.engines [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118232) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.327576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writer thread running {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.349794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.context [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0b3c262b-14d7-4ecd-824a-690cdc4a9b4a(cell1) {{(pid=118233) load_cells /opt/stack/nova/nova/context.py:464}} May 24 10:12:42.353965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:12:42.355579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:12:42.356391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:12:42.358413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:12:42.360403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO dbcounter [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Registered counter for database nova_cell0 May 24 10:12:42.364490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO dbcounter [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Registered counter for database nova_cell1 May 24 10:12:42.379070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_db.sqlalchemy.engines [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.380777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_db.sqlalchemy.engines [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.384800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.context [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0b3c262b-14d7-4ecd-824a-690cdc4a9b4a(cell1) {{(pid=118232) load_cells /opt/stack/nova/nova/context.py:464}} May 24 10:12:42.386314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writer thread running {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.388955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:12:42.390599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:12:42.391399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:12:42.391537 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writer thread running {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.393381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:12:42.395421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO dbcounter [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Registered counter for database nova_cell0 May 24 10:12:42.399381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO dbcounter [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Registered counter for database nova_cell1 May 24 10:12:42.406423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_db.sqlalchemy.engines [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118232) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.408072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_db.sqlalchemy.engines [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118232) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.414693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writer thread running {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.415960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writer thread running {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.537086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO dbcounter [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Registered counter for database nova_cell0 May 24 10:12:42.544131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_db.sqlalchemy.engines [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.548996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writer thread running {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.563469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO dbcounter [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Registered counter for database nova_cell0 May 24 10:12:42.570302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_db.sqlalchemy.engines [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=118232) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 10:12:42.574682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writer thread running {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 10:12:42.875815 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: WARNING keystonemiddleware.auth_token [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:12:42.883261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: WARNING keystonemiddleware.auth_token [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:12:42.906370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: WARNING keystonemiddleware.auth_token [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:12:42.914388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: WARNING keystonemiddleware.auth_token [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:12:43.234775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: WARNING keystonemiddleware.auth_token [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:12:43.241596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: WARNING keystonemiddleware.auth_token [None req-f3e93dcd-aa97-4724-bc56-1a04f54cfdc0 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:12:43.247008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56067b0979c0 pid: 118233 (default app) May 24 10:12:43.266930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: WARNING keystonemiddleware.auth_token [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 24 10:12:43.273182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: WARNING keystonemiddleware.auth_token [None req-9d0c8f54-a643-4dac-8f17-143dd7c82828 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 24 10:12:43.278590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56067b0979c0 pid: 118232 (default app) May 24 10:13:26.680984 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=3 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.682151 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=3 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.694005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=3 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.694842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=3 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.949187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.950366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.951147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.951978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:27.735983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-407f81c4-1f61-413b-8782-07ebd4865720 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Action: 'create', calling method: .version_select at 0x7f8734312c10>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1380358039", "metadata": {"Type": "work"}}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:27.991846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fda84987-28c3-4816-829c-e7759eba399f tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Action: 'create', calling method: .version_select at 0x7f8734314ca0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-541503220"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:30.214415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-407f81c4-1f61-413b-8782-07ebd4865720 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 3.538626 May 24 10:13:30.215884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 1/1] 10.222.0.35 () {60 vars in 1155 bytes} [Tue May 24 10:13:26 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 3542 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) May 24 10:13:30.310866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fda84987-28c3-4816-829c-e7759eba399f tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 3.372397 May 24 10:13:30.311655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1/2] 10.222.0.35 () {60 vars in 1154 bytes} [Tue May 24 10:13:26 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 3375 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) May 24 10:13:30.317919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-07147751-d7ac-4ec1-8909-85fec184c211 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Calling method '.version_select at 0x7f8734164c10>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:30.427691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-163200131", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "networks": [{"uuid": "3f99edd0-4e35-46dd-8cea-9bb9556ee2d2"}]}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:30.596618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-07147751-d7ac-4ec1-8909-85fec184c211 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1" status: 200 len: 353 microversion: 2.1 time: 0.280510 May 24 10:13:30.600594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 2/3] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:30 2022] GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 => generated 353 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:30.607183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-12c1e249-793b-4078-b399-409375dcb395 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f87341763a0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:30.677954 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] validate_networks() for [('3f99edd0-4e35-46dd-8cea-9bb9556ee2d2', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:13:31.157034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-12c1e249-793b-4078-b399-409375dcb395 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 200 len: 344 microversion: 2.1 time: 0.554924 May 24 10:13:31.157818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 3/4] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:30 2022] GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 344 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:31.608592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd448cd4-5723-4576-bd99-c808ac520d8a tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Calling method '.version_select at 0x7f8734164dc0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:31.724005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd448cd4-5723-4576-bd99-c808ac520d8a tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1" status: 200 len: 354 microversion: 2.1 time: 0.120417 May 24 10:13:31.731541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 4/5] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:31 2022] GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 => generated 354 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:31.736800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3b102cf7-d7b0-4b14-ad70-24e95d1c7e5f tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Calling method '.version_select at 0x7f8734176430>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:31.851376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3b102cf7-d7b0-4b14-ad70-24e95d1c7e5f tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1" status: 200 len: 354 microversion: 2.1 time: 0.117819 May 24 10:13:31.851969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 5/6] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:31 2022] GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 => generated 354 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:31.869654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f7b4e3f7-b301-4c49-8e5b-8414b9938df0 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Calling method '.version_select at 0x7f8734176af0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:32.194805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f7b4e3f7-b301-4c49-8e5b-8414b9938df0 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1" status: 202 len: 0 microversion: 2.1 time: 0.329075 May 24 10:13:32.195828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 6/7] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:13:31 2022] DELETE /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 => generated 0 bytes in 331 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:13:32.204647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb64cb7b-2c86-4be8-bfdd-8858cde254a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8734176ca0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:32.312182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb64cb7b-2c86-4be8-bfdd-8858cde254a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 200 len: 345 microversion: 2.1 time: 0.111180 May 24 10:13:32.313530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 7/8] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:32 2022] GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 345 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:32.324018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c5b4b7ab-8f2b-450a-bc61-f2f0c683a438 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Calling method '.version_select at 0x7f8734176310>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:32.435214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c5b4b7ab-8f2b-450a-bc61-f2f0c683a438 tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1" status: 200 len: 353 microversion: 2.1 time: 0.120282 May 24 10:13:32.435931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 8/9] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:32 2022] GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 => generated 353 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:32.440997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef4b7ac0-4f28-482e-a635-82408835e06b tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Action: 'create', calling method: .version_select at 0x7f8734164c10>, body: {"snapshot": {"volume_id": "735d74e9-b104-4571-8bf0-6937a1ba9818", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-998586335"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:32.576880 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:13:32.577419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:13:32.577501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:13:32.577924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:13:32.578345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:13:32.646915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:32.648661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:32.652383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:32.654271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:32.771559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Getting quotas for project 4784aebbfd1d43f896df6ca9057b3251. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:13:32.799908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Getting quotas for user 13f9b782a79f48678f297d1ef388da75 and project 4784aebbfd1d43f896df6ca9057b3251. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:13:32.833510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:13:32.836659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:13:33.086960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d719dd32-247f-45d8-afa2-a96df622afc0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.869642 May 24 10:13:33.095585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 2/10] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:13:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 2876 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:13:33.099448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-89f90a74-e6ff-4097-8947-6a32f67e1453 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:33.106111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-89f90a74-e6ff-4097-8947-6a32f67e1453 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:33.248774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-89f90a74-e6ff-4097-8947-6a32f67e1453 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:33.278226 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-89f90a74-e6ff-4097-8947-6a32f67e1453 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1214 microversion: 2.1 time: 0.183714 May 24 10:13:33.278817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 3/11] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:33 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1214 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:33.448303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-11a43339-e9f4-4d2c-806d-41630f3ccbfe tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Calling method '.version_select at 0x7f873301ba60>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:33.491794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef4b7ac0-4f28-482e-a635-82408835e06b tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 1.054581 May 24 10:13:33.493652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 9/12] 10.222.0.35 () {60 vars in 1161 bytes} [Tue May 24 10:13:32 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 1057 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:33.504704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-573da715-a5e0-41e3-b85a-b8bfac3a4d73 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8734af11f0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:33.663826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-11a43339-e9f4-4d2c-806d-41630f3ccbfe tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] HTTP exception thrown: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-11a43339-e9f4-4d2c-806d-41630f3ccbfe tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Request method failure captured: May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 HTTP/1.1 May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_GqbXbLirbp6KA--WexmvYe30tvq9Krfgqh5U-csR-L0ZN2nvQ-Yh4O5p_8V6HalHmY_LanccaKNmFUoChtYsSKFxvKo1-mEUCYgaCHVAu1L81bhiDTAS5lRsXB44Gzqy5aV0dtiejRUdp_KTxnv5l1HW0fSVc5mo4-JqUu9lUzq8 May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 2cac460d93d1472194ce4c01ca7fb7ca May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesGetTestJSON-1486106178 May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:13:33.670713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/2cac460d93d1472194ce4c01ca7fb7ca"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_2cac460d93d1472194ce4c01ca7fb7ca"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/2cac460d93d1472194ce4c01ca7fb7ca"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/2cac460d93d1472194ce4c01ca7fb7ca"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesGetTestJSON-1486106178 May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 2cac460d93d1472194ce4c01ca7fb7ca May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesGetTestJSON-1486106178 May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesGetTestJSON-1486106178-project May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 0716b07086384fa1bf9e8ca107603436 May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesGetTestJSON-1486106178-project May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': 'b29ba232-2fac-451a-9e20-f57b2be549b1'} May 24 10:13:33.671835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. (HTTP 404) (Request-ID: req-14a575c0-290f-42f1-a582-479857e1a47b) May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:33.672774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:33.673885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 24 10:13:33.674817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. May 24 10:13:33.676368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:33.678259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-11a43339-e9f4-4d2c-806d-41630f3ccbfe tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] Returning 404 to user: Volume b29ba232-2fac-451a-9e20-f57b2be549b1 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:13:33.685680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-573da715-a5e0-41e3-b85a-b8bfac3a4d73 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 200 len: 286 microversion: 2.1 time: 0.180023 May 24 10:13:33.685955 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-11a43339-e9f4-4d2c-806d-41630f3ccbfe tempest-VolumesGetTestJSON-1486106178 tempest-VolumesGetTestJSON-1486106178-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1" status: 404 len: 109 microversion: 2.1 time: 0.235087 May 24 10:13:33.686037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 10/13] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:33 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 286 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:13:33.686123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 4/14] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:33 2022] GET /compute/v2.1/os-volumes/b29ba232-2fac-451a-9e20-f57b2be549b1 => generated 109 bytes in 236 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:13:33.694319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6dd9ee94-106a-4e97-884c-6a07f95735ee tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8734176430>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:33.879107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6dd9ee94-106a-4e97-884c-6a07f95735ee tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.193362 May 24 10:13:33.880286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 11/15] 10.222.0.35 () {58 vars in 1139 bytes} [Tue May 24 10:13:33 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:33.894127 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-481c1f8d-3171-45d5-a6bc-cf5768418de3 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8734ae1310>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:33.987107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-481c1f8d-3171-45d5-a6bc-cf5768418de3 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 200 len: 286 microversion: 2.1 time: 0.100495 May 24 10:13:33.990651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 5/16] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:33 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 286 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:13:34.298132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cc4cf5d9-5d14-4da4-a16b-375f9824548c tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:34.299005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cc4cf5d9-5d14-4da4-a16b-375f9824548c tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:34.484140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cc4cf5d9-5d14-4da4-a16b-375f9824548c tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:34.516697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cc4cf5d9-5d14-4da4-a16b-375f9824548c tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1214 microversion: 2.1 time: 0.222548 May 24 10:13:34.519927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 12/17] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:34 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1214 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:35.002592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2d64650d-d38c-4eca-a772-6051a33a0e58 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8733876b80>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:35.043079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2d64650d-d38c-4eca-a772-6051a33a0e58 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 200 len: 286 microversion: 2.1 time: 0.044394 May 24 10:13:35.043861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 6/18] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:34 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 286 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:35.533927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96005984-3efa-4044-b477-72e9283cfaa6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:35.535089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96005984-3efa-4044-b477-72e9283cfaa6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:35.658003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96005984-3efa-4044-b477-72e9283cfaa6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:35.668025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96005984-3efa-4044-b477-72e9283cfaa6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1214 microversion: 2.1 time: 0.138143 May 24 10:13:35.670451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 13/19] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:35 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1214 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:13:36.062002 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-626f6b5f-13f8-44fc-af87-af95168d1642 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9b5e0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:36.277043 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-626f6b5f-13f8-44fc-af87-af95168d1642 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 200 len: 286 microversion: 2.1 time: 0.220615 May 24 10:13:36.277043 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 7/20] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:36 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 286 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:36.686389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:36.688315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:36.699347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:36.702321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:36.704501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:36.705214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:37.026441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:37.052449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96bf6a85-cc0b-45b3-b09b-82bab3fc19b5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1218 microversion: 2.1 time: 0.369786 May 24 10:13:37.053488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 14/21] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:36 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1218 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:37.286461 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2fbc8b91-42d8-4bc9-8b8e-624a13726301 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9bca0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:37.351442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2fbc8b91-42d8-4bc9-8b8e-624a13726301 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 200 len: 287 microversion: 2.1 time: 0.069481 May 24 10:13:37.352164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 8/22] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:37 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 287 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:37.362130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e9f84055-9e70-4bc8-bb2c-4688b859336a tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f87340cb5e0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:37.553505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e9f84055-9e70-4bc8-bb2c-4688b859336a tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "DELETE /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 202 len: 0 microversion: 2.1 time: 0.195079 May 24 10:13:37.554438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 15/23] 10.222.0.35 () {58 vars in 1253 bytes} [Tue May 24 10:13:37 2022] DELETE /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 0 bytes in 197 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:13:37.563132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bd867c2d-c712-4d52-9023-d6e0c2ed67fe tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9baf0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:37.634588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bd867c2d-c712-4d52-9023-d6e0c2ed67fe tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 200 len: 286 microversion: 2.1 time: 0.076182 May 24 10:13:37.637673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 9/24] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:37 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 286 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:38.069548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:38.070929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:38.081191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:38.088585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:38.090033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:38.090523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:38.288939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:38.299860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-37626e3e-08c8-4065-a0d1-5cbe8592c0f2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1210 microversion: 2.1 time: 0.234700 May 24 10:13:38.305872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 16/25] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:38 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1210 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:38.649721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d13368bc-4ac6-4ceb-a19c-b6ff4c071a94 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9b430>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:38.719368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-d13368bc-4ac6-4ceb-a19c-b6ff4c071a94 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] HTTP exception thrown: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d13368bc-4ac6-4ceb-a19c-b6ff4c071a94 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Request method failure captured: May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b HTTP/1.1 May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_GvMe8XB6UuBDreveoHQmR99CjsXOaxzj8PHwD8QMATOm0Q1-Bw0YRMx5NBeVhKsxbDPXmjI16h0dvqJcEVlPk3etexAg9nIwZ0WTPfHc8govvqqQn_qcdSxeDnMZi8UGU7Oq0co2l1za4Y0QRnk_lmmjnSeKz9TcQBpcKZUF6UtE May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:13:38.722007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: e59aab502a9a43189637822d3dd2be57 May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b'} May 24 10:13:38.723179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. (HTTP 404) (Request-ID: req-edb61d1a-e7d2-419c-b15e-3bc88bcdb52e) May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.723891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:38.724974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show May 24 10:13:38.725928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d13368bc-4ac6-4ceb-a19c-b6ff4c071a94 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Returning 404 to user: Snapshot 3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:13:38.726936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d13368bc-4ac6-4ceb-a19c-b6ff4c071a94 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b" status: 404 len: 111 microversion: 2.1 time: 0.080509 May 24 10:13:38.733882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-851ffc2c-c166-447d-8c75-a4a0386c07d7 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8734075940>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:38.738378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 10/26] 10.222.0.35 () {58 vars in 1250 bytes} [Tue May 24 10:13:38 2022] GET /compute/v2.1/os-snapshots/3072ce59-2a51-4cb4-9a9d-f78d3bf0aa3b => generated 111 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:13:39.135164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-851ffc2c-c166-447d-8c75-a4a0386c07d7 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 202 len: 0 microversion: 2.1 time: 0.405767 May 24 10:13:39.148438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 17/27] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:13:38 2022] DELETE /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 0 bytes in 420 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:13:39.148839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-33afe63b-2f4b-4614-9333-0b836c8f61bc tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f873302daf0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:39.317451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-33afe63b-2f4b-4614-9333-0b836c8f61bc tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 200 len: 344 microversion: 2.1 time: 0.176941 May 24 10:13:39.318474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 11/28] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:39 2022] GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:13:39.326657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:39.327289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:39.344319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:39.345976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:39.347712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:39.348590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:39.546513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:39.559162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-047d4d75-5398-4ec9-a364-e4bfdef953c8 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1274 microversion: 2.1 time: 0.242167 May 24 10:13:39.564032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 18/29] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:39 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1274 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:40.059927 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-152e559f-325a-43ff-bc25-7a40b1025f64 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'create', calling method: .version_select at 0x7f8734ae10d0>, body: {"keypair": {"name": "tempest-keypair-392535260", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKTI9fyAnmaGrYJAzKhuc3oVglTo3Q6mzdoG4Dtpg4N6AZE6xKOt6dxbZPOMg9H84T9bd6yzcnUHF95yFi4+0p7SdvvFNZbvZnERHxaeND2s4COqkOP6YBWdjxQAFzitrw=="}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:40.135339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-152e559f-325a-43ff-bc25-7a40b1025f64 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'key_pairs'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:13:40.142631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-152e559f-325a-43ff-bc25-7a40b1025f64 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for user 7bffd469507b4dbc8d3146f051c8c8ad and project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'key_pairs'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:13:40.394516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-152e559f-325a-43ff-bc25-7a40b1025f64 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.740695 May 24 10:13:40.403471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 12/30] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:13:39 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 754 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:40.409116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8b419436-a9c2-4c10-a2a1-8e007fd2cdec tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9a3a0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:40.479191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-8b419436-a9c2-4c10-a2a1-8e007fd2cdec tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] HTTP exception thrown: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8b419436-a9c2-4c10-a2a1-8e007fd2cdec tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Request method failure captured: May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 HTTP/1.1 May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_GvMe8XB6UuBDreveoHQmR99CjsXOaxzj8PHwD8QMATOm0Q1-Bw0YRMx5NBeVhKsxbDPXmjI16h0dvqJcEVlPk3etexAg9nIwZ0WTPfHc8govvqqQn_qcdSxeDnMZi8UGU7Oq0co2l1za4Y0QRnk_lmmjnSeKz9TcQBpcKZUF6UtE May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:13:40.482450 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: e59aab502a9a43189637822d3dd2be57 May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '735d74e9-b104-4571-8bf0-6937a1ba9818'} May 24 10:13:40.488130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. (HTTP 404) (Request-ID: req-10b979fa-8d04-47d3-aeec-215eff65a221) May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:40.489365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:40.490263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 24 10:13:40.491104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8b419436-a9c2-4c10-a2a1-8e007fd2cdec tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Returning 404 to user: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:13:40.491987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8b419436-a9c2-4c10-a2a1-8e007fd2cdec tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 404 len: 109 microversion: 2.1 time: 0.079572 May 24 10:13:40.494851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 13/31] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:40 2022] GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 109 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:13:40.501553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cfe29d12-1fbd-45e5-82e6-fe2dc299b20e tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9a670>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:40.549524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-cfe29d12-1fbd-45e5-82e6-fe2dc299b20e tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] HTTP exception thrown: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cfe29d12-1fbd-45e5-82e6-fe2dc299b20e tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Request method failure captured: May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: DELETE /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 HTTP/1.1 May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_GvMe8XB6UuBDreveoHQmR99CjsXOaxzj8PHwD8QMATOm0Q1-Bw0YRMx5NBeVhKsxbDPXmjI16h0dvqJcEVlPk3etexAg9nIwZ0WTPfHc8govvqqQn_qcdSxeDnMZi8UGU7Oq0co2l1za4Y0QRnk_lmmjnSeKz9TcQBpcKZUF6UtE May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:13:40.557382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: e59aab502a9a43189637822d3dd2be57 May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '735d74e9-b104-4571-8bf0-6937a1ba9818'} May 24 10:13:40.558583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. (HTTP 404) (Request-ID: req-dd071ac2-8cd2-418a-8bc0-8545b1afa54a) May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:40.559242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 24 10:13:40.560100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 24 10:13:40.561040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:40.561968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:40.570234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cfe29d12-1fbd-45e5-82e6-fe2dc299b20e tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Returning 404 to user: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:13:40.571100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cfe29d12-1fbd-45e5-82e6-fe2dc299b20e tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 404 len: 109 microversion: 2.1 time: 0.074990 May 24 10:13:40.576760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 14/32] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:13:40 2022] DELETE /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 109 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:13:40.591081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:40.593956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:40.611774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:40.612508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:40.613469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:40.614559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:40.874203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2037157838", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:41.101320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] validate_networks() for [('d063cd42-5325-4a1c-ac6c-ec74b4b895ae', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:13:41.259878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:41.289987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-13beadd5-e7d3-4ea9-8757-48c9c60679ea tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1272 microversion: 2.1 time: 0.710989 May 24 10:13:41.294902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 15/33] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:40 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1272 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:41.308602 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ad57ed7d-a69b-44cf-b033-33164e5dc2a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Calling method '.version_select at 0x7f8732f9aca0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:41.374260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-ad57ed7d-a69b-44cf-b033-33164e5dc2a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] HTTP exception thrown: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ad57ed7d-a69b-44cf-b033-33164e5dc2a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Request method failure captured: May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 HTTP/1.1 May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_GvMe8XB6UuBDreveoHQmR99CjsXOaxzj8PHwD8QMATOm0Q1-Bw0YRMx5NBeVhKsxbDPXmjI16h0dvqJcEVlPk3etexAg9nIwZ0WTPfHc8govvqqQn_qcdSxeDnMZi8UGU7Oq0co2l1za4Y0QRnk_lmmjnSeKz9TcQBpcKZUF6UtE May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:13:41.376520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/35a9ad5293a844f4a481f73dcee4f86b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 35a9ad5293a844f4a481f73dcee4f86b May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-296677629 May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: e59aab502a9a43189637822d3dd2be57 May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesSnapshotsTestJSON-296677629-project May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '735d74e9-b104-4571-8bf0-6937a1ba9818'} May 24 10:13:41.378327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. (HTTP 404) (Request-ID: req-61950326-c7e6-4ea0-990a-17aaccedcfe5) May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:41.379199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:13:41.380298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 24 10:13:41.381419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. May 24 10:13:41.382524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:13:41.384095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ad57ed7d-a69b-44cf-b033-33164e5dc2a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] Returning 404 to user: Volume 735d74e9-b104-4571-8bf0-6937a1ba9818 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:13:41.385113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ad57ed7d-a69b-44cf-b033-33164e5dc2a6 tempest-VolumesSnapshotsTestJSON-296677629 tempest-VolumesSnapshotsTestJSON-296677629-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818" status: 404 len: 109 microversion: 2.1 time: 0.091701 May 24 10:13:41.388985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 16/34] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:13:41 2022] GET /compute/v2.1/os-volumes/735d74e9-b104-4571-8bf0-6937a1ba9818 => generated 109 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:13:42.312145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:42.313643 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:42.350309 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:42.465682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:42.465682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:42.465682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:42.551149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:42.568905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b9347a40-3323-4586-a895-ee4c2bc80c49 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.260757 May 24 10:13:42.570317 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 17/35] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:42 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:43.593101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:43.593956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:43.596070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:43.613596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:43.618895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:43.620288 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:43.629775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:44.275454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:44.315533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6e584176-7ef2-4513-ad54-d9d007d9d08a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.725817 May 24 10:13:44.324128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 18/36] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:43 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:13:45.339438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:45.340491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:45.353196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:45.354396 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:45.355483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:45.358338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:45.664701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:45.705016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:13:45.707115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:13:45.707931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:13:45.708501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:13:45.709076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:13:45.749648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:45.750281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:45.751399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:45.751995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:45.889745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:13:45.915653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for user baac9b5e923747e6986720e3ac849056 and project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:13:45.968987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:13:45.981891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:13:46.266600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e7575251-96a5-4d82-83cb-43c29b55d357 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 6.049068 May 24 10:13:46.280942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 19/37] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:13:40 2022] POST /compute/v2.1/servers => generated 383 bytes in 6064 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:13:46.307329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-86ef856f-366e-4f6e-8422-fab7a3670254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:46.309345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-86ef856f-366e-4f6e-8422-fab7a3670254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:46.474669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-86ef856f-366e-4f6e-8422-fab7a3670254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:46.485366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-86ef856f-366e-4f6e-8422-fab7a3670254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1222 microversion: 2.1 time: 0.197525 May 24 10:13:46.491778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 20/38] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:46 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1222 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:46.525887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:46.550769 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-89321807-d670-4348-96e0-566e470a63c2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 1.209294 May 24 10:13:46.555244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 19/39] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:45 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 1219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:13:46.793010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1634465282", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:46.928646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] validate_networks() for [('d063cd42-5325-4a1c-ac6c-ec74b4b895ae', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:13:47.571321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:47.578990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:47.592017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:47.592495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:47.595973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:47.595973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:47.876655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:13:47.877229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:13:47.878804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:13:47.879775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:13:47.880406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:13:47.910927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:47.913537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:47.918148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:47.918148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:47.982032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:13:47.992149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for user baac9b5e923747e6986720e3ac849056 and project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:13:48.030207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:13:48.035811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:13:48.296723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:48.310290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9a061f51-c726-4f40-9680-33d31a5f5dec tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.742249 May 24 10:13:48.318852 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 20/40] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:47 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:48.337257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1490803591", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c3adfbde-72f1-4452-b19b-552144101170"}], "security_groups": [{"name": "tempest-securitygroup--339617936"}], "key_name": "tempest-keypair-392535260", "user_data": "}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:48.351165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3f959c96-133d-44da-a5f8-e7b736d7e7d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.562571 May 24 10:13:48.357032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 21/41] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:13:46 2022] POST /compute/v2.1/servers => generated 383 bytes in 1569 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:13:48.382080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a5278f79-744d-41d8-8e19-71e425072d18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:48.383468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a5278f79-744d-41d8-8e19-71e425072d18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:48.662042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a5278f79-744d-41d8-8e19-71e425072d18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:48.679891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a5278f79-744d-41d8-8e19-71e425072d18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1222 microversion: 2.1 time: 0.316059 May 24 10:13:48.682219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 22/42] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:48 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1222 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:48.874425 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] validate_networks() for [('c3adfbde-72f1-4452-b19b-552144101170', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:13:49.649758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-44ec5787-488d-4fff-9f1a-fa36ccdc1934 req-f5723c67-de73-4b2c-8bc9-283d523a38df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d750c8ea-4be5-482e-957b-50352a34a21f", "tag": "d1b0d253-6c27-4bcb-b946-972069c56a73"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:49.715144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-44ec5787-488d-4fff-9f1a-fa36ccdc1934 req-f5723c67-de73-4b2c-8bc9-283d523a38df service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:49.738951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-44ec5787-488d-4fff-9f1a-fa36ccdc1934 req-f5723c67-de73-4b2c-8bc9-283d523a38df service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:49.785484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-44ec5787-488d-4fff-9f1a-fa36ccdc1934 req-f5723c67-de73-4b2c-8bc9-283d523a38df service nova] Creating event network-changed:d1b0d253-6c27-4bcb-b946-972069c56a73 for instance d750c8ea-4be5-482e-957b-50352a34a21f on hv-cinder-84311 May 24 10:13:49.787490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-44ec5787-488d-4fff-9f1a-fa36ccdc1934 req-f5723c67-de73-4b2c-8bc9-283d523a38df service nova] Acquired lock "compute-rpcapi-router" {{(pid=118232) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 24 10:13:49.791987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:49.792859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:49.794237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:49.797981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:49.827366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.compute.rpcapi [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 24 10:13:49.831509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] Releasing lock "compute-rpcapi-router" {{(pid=118232) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 24 10:13:49.874422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7e89c9c7-2414-4292-a6cd-55a51ac27286 None None] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.795734 May 24 10:13:49.875518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 23/43] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:13:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:49.895607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:49.896318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:49.929305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:49.929787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:49.930571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:49.938111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:50.173192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=12 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:50.264590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:50.275754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a99694dc-80e7-4f24-a26e-2f546af3a8fe tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.398768 May 24 10:13:50.276785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 24/44] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:49 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:50.297523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:50.301811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:50.320472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:50.321197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:50.322239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:50.322825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:50.482247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:50.493425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a6b3c88e-ea59-45b2-8506-c11009ff2e76 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1226 microversion: 2.1 time: 0.215216 May 24 10:13:50.504806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 25/45] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:50 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1226 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:50.531902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:13:50.532414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:13:50.532952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:13:50.534600 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:13:50.535237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:13:50.550352 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:50.551980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:50.553659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:50.555163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:50.589514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:13:50.595439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for user 7bffd469507b4dbc8d3146f051c8c8ad and project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:13:50.608473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:13:50.610721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:13:50.744748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8bec4311-95ec-441a-88ef-955dc78ffa85 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.424674 May 24 10:13:50.745717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 21/46] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:13:48 2022] POST /compute/v2.1/servers => generated 401 bytes in 2426 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:13:50.757979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5984a640-56fe-4ce5-95b2-c8840b85036a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:50.759372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5984a640-56fe-4ce5-95b2-c8840b85036a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:50.855919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5984a640-56fe-4ce5-95b2-c8840b85036a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:50.870612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5984a640-56fe-4ce5-95b2-c8840b85036a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1237 microversion: 2.1 time: 0.117862 May 24 10:13:50.871751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 26/47] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:50 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1237 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:51.293709 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:51.294327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:51.303688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:51.304148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:51.304937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:51.305405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:51.464291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:51.475281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bf3f4f04-8b36-4277-9e45-52482eb12e45 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.185523 May 24 10:13:51.475861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 22/48] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:51 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:51.519022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:51.520047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:51.535265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:51.540050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:51.541412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:51.542306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:51.653833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:51.663184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-83d58876-0b9e-46b1-9fc8-ebfb7f6d2633 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1226 microversion: 2.1 time: 0.149319 May 24 10:13:51.665333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 27/49] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:51 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1226 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:51.894448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d8281396-7236-4019-9b3e-0af4694de60a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:51.895088 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d8281396-7236-4019-9b3e-0af4694de60a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:52.109357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d8281396-7236-4019-9b3e-0af4694de60a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:52.137779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d8281396-7236-4019-9b3e-0af4694de60a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1237 microversion: 2.1 time: 0.249328 May 24 10:13:52.138445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 23/50] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:51 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1237 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:52.493517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:52.494283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:52.510704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:52.511894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:52.513170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:52.514981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:52.680003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:52.680632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:52.688821 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:52.689354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:52.690163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:52.694999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:52.839869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:52.855507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4ad35b7d-90d3-4979-aa38-79e219a813b2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.365679 May 24 10:13:52.856103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 28/51] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:52 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:52.905232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:52.919995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6349a74b-5d93-4766-a4e5-4d3008623a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1274 microversion: 2.1 time: 0.243473 May 24 10:13:52.927093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 24/52] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:52 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1274 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:53.155440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:53.157585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:53.191581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:53.193389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:53.195142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:53.196936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:53.304995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:53.314701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-80ccd64f-fd1d-420b-b782-168a24110f8d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1233 microversion: 2.1 time: 0.163202 May 24 10:13:53.316289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 29/53] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:53 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1233 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:53.877583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:53.879739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:53.887797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:53.888585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:53.889682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:53.890378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:53.962823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:53.966155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:53.981141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:53.997158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:53.999559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:54.005579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:54.098723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:54.111491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-792d18c9-22db-48cc-b387-0ad30f6a0c70 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.224064 May 24 10:13:54.111491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 25/54] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:53 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:54.152071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:54.159933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9dce7d1c-5f97-4456-b4eb-85f7b25a8c61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1274 microversion: 2.1 time: 0.214592 May 24 10:13:54.170606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 30/55] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:53 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1274 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:54.332908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:54.333567 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:54.339679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:54.340129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:54.340896 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:54.341374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:54.407839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:54.416141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-91e5239e-487b-424b-bda5-85e6263e6c2e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1233 microversion: 2.1 time: 0.087042 May 24 10:13:54.421720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 26/56] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:54 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1233 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:55.129685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:55.131077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:55.144407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:55.145097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:55.146132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:55.146797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:55.178658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:55.180484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:55.197654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:55.199272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:55.201403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:55.202560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:55.371639 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:55.382126 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2a2f5b4d-144a-44aa-97c4-cb82d05cb777 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1282 microversion: 2.1 time: 0.208649 May 24 10:13:55.391168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 27/57] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:55 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1282 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:55.433688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:55.434325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:55.442386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:55.442872 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:55.443646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:55.444092 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:55.529180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:55.536965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:55.539142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9ded27ff-36a4-4e1c-9c57-c87095c072e5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.414974 May 24 10:13:55.542920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 31/58] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:55 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:55.545231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6c54e9aa-5286-4365-a74b-0eafacd477f2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1289 microversion: 2.1 time: 0.115345 May 24 10:13:55.546011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 28/59] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:55 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1289 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:13:56.409593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:56.410068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:56.417535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:56.418004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:56.418770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:56.419370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:56.535995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:56.558374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:56.559616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:56.566218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:56.566876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:56.581707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-40a97a3e-a36b-4537-9880-a3f57cf0cdec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1282 microversion: 2.1 time: 0.161296 May 24 10:13:56.581707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 32/60] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:56 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1282 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:56.582202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:56.582202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:56.582382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:56.582382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:56.588529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:56.588994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:56.590195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:56.590644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:56.707900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:56.717526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d2fde919-f6e6-413a-ae74-9bac25aa2f72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1289 microversion: 2.1 time: 0.147044 May 24 10:13:56.718353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 33/61] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:56 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1289 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:56.859773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:56.868971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-13312ab2-84da-4197-85ed-63f56cb94c63 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.316360 May 24 10:13:56.880959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 29/62] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:56 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:57.593198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:57.594710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:57.599063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=6 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:57.613223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:57.614040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:57.615620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:57.616227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:57.735696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:57.736312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:57.740400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=7 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:57.754054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:57.754772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:57.755742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:57.756338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:57.761604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=77 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:57.932426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:57.946201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e1b6e7b6-1745-4410-b012-bdaa1cbde1bc tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1295 microversion: 2.1 time: 0.214274 May 24 10:13:57.947102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 30/63] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:57 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1295 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:13:57.956195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:57.957027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:57.969129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:57.969853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:57.972061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:57.972659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:58.016858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:58.025708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9a1a088c-eb02-412a-9d20-9a2d2716e4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.440138 May 24 10:13:58.036438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 34/64] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:57 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:13:58.171196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-1e11f2d9-14a5-4a07-95d0-8724c8176c10 req-b7122283-0554-4435-baea-d406581c5ea6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b3484367-cf43-4128-b7f2-96124543234c", "tag": "0241cf1e-f6a3-46ac-b0d9-a787b63f4fcf"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:13:58.179119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-1e11f2d9-14a5-4a07-95d0-8724c8176c10 req-b7122283-0554-4435-baea-d406581c5ea6 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:58.179561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-1e11f2d9-14a5-4a07-95d0-8724c8176c10 req-b7122283-0554-4435-baea-d406581c5ea6 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:58.184369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=87 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:58.193385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-1e11f2d9-14a5-4a07-95d0-8724c8176c10 req-b7122283-0554-4435-baea-d406581c5ea6 service nova] Creating event network-changed:0241cf1e-f6a3-46ac-b0d9-a787b63f4fcf for instance b3484367-cf43-4128-b7f2-96124543234c on hv-cinder-84311 May 24 10:13:58.199220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-1e11f2d9-14a5-4a07-95d0-8724c8176c10 req-b7122283-0554-4435-baea-d406581c5ea6 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.137183 May 24 10:13:58.200315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 35/65] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:13:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:13:58.461713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:58.473311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-41add3d1-2b0b-4ebe-9f14-1ba73fffa7f0 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.524886 May 24 10:13:58.482563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 31/66] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:57 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:58.972777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:58.975439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:58.990700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:58.992766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:58.993845 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:58.994489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:59.043566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:59.045154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:59.056150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:59.056601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:59.057409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:59.057846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:59.087038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:59.095172 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-04896001-b3ae-419f-af3a-3abc164f24f6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1295 microversion: 2.1 time: 0.128116 May 24 10:13:59.096371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 36/67] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:58 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1295 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:59.483874 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:59.493813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-70549a13-753c-453e-bdfb-8a4b74dcc9e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.454641 May 24 10:13:59.502642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:13:59.505856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:13:59.517875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 32/68] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:59 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:13:59.518442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:59.519141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:59.520868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:13:59.521552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:13:59.821364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=6 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:59.849860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:13:59.857685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-58b36d41-20fc-45d9-8610-4162bb6b1f0f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.361273 May 24 10:13:59.858991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 37/69] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:13:59 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:00.113436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:00.114653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:00.121214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.121696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.122462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.123030 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.414902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:00.430137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4d241fae-fc17-4e25-867f-d443a81515a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.321654 May 24 10:14:00.439639 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 33/70] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:00 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:00.535313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:00.536232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:00.554999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.554999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.554999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.554999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.555353 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-c2b0eada-b6fa-4f4c-8513-6246f3d50ac0 req-01c804d2-b824-45f4-8f81-68074d4fc983 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "75b93292-1e43-43c3-913b-eee6a6623e73", "tag": "da6486e7-8d1e-46c5-abb7-88003d512083"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:00.574696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-c2b0eada-b6fa-4f4c-8513-6246f3d50ac0 req-01c804d2-b824-45f4-8f81-68074d4fc983 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.575455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-c2b0eada-b6fa-4f4c-8513-6246f3d50ac0 req-01c804d2-b824-45f4-8f81-68074d4fc983 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.606160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:00.616792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-c2b0eada-b6fa-4f4c-8513-6246f3d50ac0 req-01c804d2-b824-45f4-8f81-68074d4fc983 service nova] Creating event network-changed:da6486e7-8d1e-46c5-abb7-88003d512083 for instance 75b93292-1e43-43c3-913b-eee6a6623e73 on hv-cinder-84311 May 24 10:14:00.617595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-c2b0eada-b6fa-4f4c-8513-6246f3d50ac0 req-01c804d2-b824-45f4-8f81-68074d4fc983 service nova] Acquired lock "compute-rpcapi-router" {{(pid=118233) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 24 10:14:00.628719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.629406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.630936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.631536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.649522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.compute.rpcapi [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] Automatically selected compute RPC version 6.0 from minimum service version 61 May 24 10:14:00.650884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] Releasing lock "compute-rpcapi-router" {{(pid=118233) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 24 10:14:00.678595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c25068f2-95b0-48e2-bca9-9c97995f0acf None None] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.134479 May 24 10:14:00.679483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 34/71] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:14:00.871314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=18 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:00.879316 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:00.880261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:00.899913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.899913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.904011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:00.905390 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:00.948157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:00.957699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a7d36791-19c1-46ee-8f0a-dffebd87eb17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.430769 May 24 10:14:00.963471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 38/72] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:00 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:01.298756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:01.324099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f7e909bf-c154-4642-adb0-36293e76f312 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.452100 May 24 10:14:01.332290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 35/73] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:00 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:01.448576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:01.449317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:01.463328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:01.465776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:01.466584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:01.468221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:01.720098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:01.749370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6e0e66b3-9775-48d5-949e-c060f230eab0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.305790 May 24 10:14:01.750075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 39/74] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:01 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:01.979618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:01.980238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:01.987313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:01.988078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:01.988870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:01.989338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:02.132716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=8 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:02.136374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:02.145104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fa300c53-c2dc-4c51-9d16-0ccab23a43f8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.169378 May 24 10:14:02.145954 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 36/75] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:01 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:02.349721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:02.350335 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:02.356641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:02.359834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:02.359834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:02.359834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:02.589618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-f90f4a9e-2798-489a-8991-80367848ae4e req-ab68c547-06f0-47c4-95cd-aae6f97d8f2d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d750c8ea-4be5-482e-957b-50352a34a21f", "name": "network-vif-plugged", "status": "completed", "tag": "d1b0d253-6c27-4bcb-b946-972069c56a73"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:02.604655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-f90f4a9e-2798-489a-8991-80367848ae4e req-ab68c547-06f0-47c4-95cd-aae6f97d8f2d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:02.606090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-f90f4a9e-2798-489a-8991-80367848ae4e req-ab68c547-06f0-47c4-95cd-aae6f97d8f2d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:02.631653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-f90f4a9e-2798-489a-8991-80367848ae4e req-ab68c547-06f0-47c4-95cd-aae6f97d8f2d service nova] Creating event network-vif-plugged:d1b0d253-6c27-4bcb-b946-972069c56a73 for instance d750c8ea-4be5-482e-957b-50352a34a21f on hv-cinder-84311 May 24 10:14:02.640732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-f90f4a9e-2798-489a-8991-80367848ae4e req-ab68c547-06f0-47c4-95cd-aae6f97d8f2d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.518165 May 24 10:14:02.642429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 40/76] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:02.691113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:02.701087 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9b2ad0eb-558a-4aff-bc22-74822bb1555a tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.357413 May 24 10:14:02.702355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 37/77] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:02 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:02.722299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-22a5bfe9-1500-4903-b6ee-0cfc0d2b362c req-80cbae48-f133-4c1a-99a2-afa0e857d739 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ca33d7e-37bd-4d91-8a41-866155df1bc3", "tag": "697741fe-a360-4dd6-b2c9-a761d441a98d"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:02.744312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-22a5bfe9-1500-4903-b6ee-0cfc0d2b362c req-80cbae48-f133-4c1a-99a2-afa0e857d739 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:02.744745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-22a5bfe9-1500-4903-b6ee-0cfc0d2b362c req-80cbae48-f133-4c1a-99a2-afa0e857d739 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:02.756998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-22a5bfe9-1500-4903-b6ee-0cfc0d2b362c req-80cbae48-f133-4c1a-99a2-afa0e857d739 service nova] Creating event network-changed:697741fe-a360-4dd6-b2c9-a761d441a98d for instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 on hv-cinder-84311 May 24 10:14:02.764928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-22a5bfe9-1500-4903-b6ee-0cfc0d2b362c req-80cbae48-f133-4c1a-99a2-afa0e857d739 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048570 May 24 10:14:02.767099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 41/78] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:02.769743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:02.770654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:02.778462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:02.779377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:02.780515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:02.781488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:03.064259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:03.074414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8d273bd2-4891-4a69-bcd4-ea61443aa925 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.312180 May 24 10:14:03.082386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 38/79] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:02 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:03.170286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:03.171119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:03.178616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:03.179124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:03.179958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:03.180440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:03.367013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:03.375354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c75ed939-c3ab-471b-9d32-33dfa00408ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.215976 May 24 10:14:03.375842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 42/80] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:03 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:03.718772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:03.719374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:03.726430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:03.726880 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:03.727782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:03.728218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:03.933910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:03.940139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-48ba2dfc-8d26-41ec-acd9-d98facf344cb tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.225084 May 24 10:14:03.940719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 39/81] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:03 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:04.091285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:04.091848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:04.098141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.098890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:04.099914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.100449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:04.278133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:04.286162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0312dbb3-301b-4a21-828b-f418a07af597 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.198866 May 24 10:14:04.286739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 43/82] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:04 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:04.391789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:04.393503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:04.402041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.402498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:04.403274 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.403712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:04.597597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:04.608948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9d393881-2730-4bf1-bd49-6ff30aff47c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.218541 May 24 10:14:04.608948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 40/83] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:04 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:04.658350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-5db215f0-0113-4cff-b9b6-7c12b2e8e2fa req-b2f0d274-df5c-4e17-86c4-eaee07b9985d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b3484367-cf43-4128-b7f2-96124543234c", "name": "network-vif-plugged", "status": "completed", "tag": "0241cf1e-f6a3-46ac-b0d9-a787b63f4fcf"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:04.668823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5db215f0-0113-4cff-b9b6-7c12b2e8e2fa req-b2f0d274-df5c-4e17-86c4-eaee07b9985d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.669521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5db215f0-0113-4cff-b9b6-7c12b2e8e2fa req-b2f0d274-df5c-4e17-86c4-eaee07b9985d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:04.682417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-5db215f0-0113-4cff-b9b6-7c12b2e8e2fa req-b2f0d274-df5c-4e17-86c4-eaee07b9985d service nova] Creating event network-vif-plugged:0241cf1e-f6a3-46ac-b0d9-a787b63f4fcf for instance b3484367-cf43-4128-b7f2-96124543234c on hv-cinder-84311 May 24 10:14:04.690407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-5db215f0-0113-4cff-b9b6-7c12b2e8e2fa req-b2f0d274-df5c-4e17-86c4-eaee07b9985d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034237 May 24 10:14:04.690407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 44/84] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:04.959001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:04.959611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:04.965578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.966041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:04.966798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:04.967241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:05.124151 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:05.132695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-13f6b8f5-6201-421e-a37d-5c24033a4c9e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1314 microversion: 2.1 time: 0.177856 May 24 10:14:05.133260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 41/85] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:04 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1314 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:05.309025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:05.309025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:05.322388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:05.323192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:05.325072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:05.325723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:05.477918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:05.485348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bc76dee2-09de-4c23-92e3-1bc57e94f6c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.185014 May 24 10:14:05.485958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 45/86] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:05 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:05.624244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:05.624924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:05.631536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:05.631996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:05.632962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:05.633579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:05.912327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:05.923167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4cfa1759-483e-4ee3-9ec9-c54e9f348ad2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.302953 May 24 10:14:05.923796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 42/87] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:05 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:06.155108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:06.155987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:06.173005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:06.175917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:06.177087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:06.179848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:06.423376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:06.433375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3ad3d091-02f6-4589-af4b-4f823b3d2ba2 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1502 microversion: 2.1 time: 0.284365 May 24 10:14:06.434548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 46/88] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:06 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1502 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:06.505948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:06.506580 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:06.513244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:06.513722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:06.514657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:06.515252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:06.769429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:06.788531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c7227ff1-a0ba-4db6-a349-3b6ef02a70c0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.286421 May 24 10:14:06.798514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 43/89] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:06 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:06.942018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:06.942647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:06.949245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:06.949728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:06.950745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:06.951477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:07.271416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:07.286963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f74ab635-74bc-4597-bb87-8d033c65d38f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.349621 May 24 10:14:07.287417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 47/90] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:06 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:07.551429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-6834fa82-286e-4e3c-a32a-833adbfb49d8 req-c6225722-ea34-4acc-9e35-b0cde0898a58 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "75b93292-1e43-43c3-913b-eee6a6623e73", "name": "network-vif-plugged", "status": "completed", "tag": "da6486e7-8d1e-46c5-abb7-88003d512083"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:07.561013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-6834fa82-286e-4e3c-a32a-833adbfb49d8 req-c6225722-ea34-4acc-9e35-b0cde0898a58 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:07.561599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-6834fa82-286e-4e3c-a32a-833adbfb49d8 req-c6225722-ea34-4acc-9e35-b0cde0898a58 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:07.573018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-6834fa82-286e-4e3c-a32a-833adbfb49d8 req-c6225722-ea34-4acc-9e35-b0cde0898a58 service nova] Creating event network-vif-plugged:da6486e7-8d1e-46c5-abb7-88003d512083 for instance 75b93292-1e43-43c3-913b-eee6a6623e73 on hv-cinder-84311 May 24 10:14:07.578112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-6834fa82-286e-4e3c-a32a-833adbfb49d8 req-c6225722-ea34-4acc-9e35-b0cde0898a58 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030449 May 24 10:14:07.578885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 44/91] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:07.805669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:07.806408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:07.812449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:07.812893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:07.813723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:07.814674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:08.064440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:08.072284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bf3619c0-624f-4d81-bdb9-4d3e03d9841f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.271168 May 24 10:14:08.072861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 48/92] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:07 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:08.303675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:08.305057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:08.311443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:08.311888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:08.312824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:08.313440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:08.475341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:08.484095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cbb3c6bf-2e5f-4271-b56a-cafa65558622 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.184565 May 24 10:14:08.495136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 45/93] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:08 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:08.504080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "196aff45-d766-4e19-bbb5-4fd489ea2749"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:08.508782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:08.512272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:08.513018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:08.513832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:08.514721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:09.091187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:09.094702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:09.100119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:09.100565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:09.101629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:09.102061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:09.355269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:09.364806 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5ca5cb9b-2cce-4b7c-baa0-62df3d0edaa5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.279279 May 24 10:14:09.365428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 46/94] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:09 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:09.507543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:09.507543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:09.510781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:09.511230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:09.511995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:09.512425 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:09.693231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:09.695673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:09.702836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-94bd0b90-2511-4341-a8b3-050c2c794fea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1322 microversion: 2.1 time: 0.204926 May 24 10:14:09.703413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 47/95] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:09 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1322 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:09.720134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-f058f4fe-f46f-496b-8b9a-0950d7cb467a req-db227d01-01b9-4be4-9781-a9db0971599d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca33d7e-37bd-4d91-8a41-866155df1bc3", "name": "network-vif-plugged", "status": "completed", "tag": "697741fe-a360-4dd6-b2c9-a761d441a98d"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:09.730336 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-f058f4fe-f46f-496b-8b9a-0950d7cb467a req-db227d01-01b9-4be4-9781-a9db0971599d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:09.730936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-f058f4fe-f46f-496b-8b9a-0950d7cb467a req-db227d01-01b9-4be4-9781-a9db0971599d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:09.744711 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-f058f4fe-f46f-496b-8b9a-0950d7cb467a req-db227d01-01b9-4be4-9781-a9db0971599d service nova] Creating event network-vif-plugged:697741fe-a360-4dd6-b2c9-a761d441a98d for instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 on hv-cinder-84311 May 24 10:14:09.749951 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-f058f4fe-f46f-496b-8b9a-0950d7cb467a req-db227d01-01b9-4be4-9781-a9db0971599d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035418 May 24 10:14:09.750554 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 48/96] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:09.875130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-322fb70e-bfa5-4a1c-a461-a2778458b35e tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "POST /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.375695 May 24 10:14:09.882582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 49/97] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:14:08 2022] POST /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f/os-volume_attachments => generated 194 bytes in 1384 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:10.382756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:10.383479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:10.389636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:10.390083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:10.390843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:10.391305 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:10.571775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:10.579296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7a627c0e-0cbd-47fa-92d3-72d1b4d3675f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.200904 May 24 10:14:10.580426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 49/98] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:10 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:10.720005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:10.721315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:10.728752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:10.729203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:10.729989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:10.730420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:10.884190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:10.898001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9739c1a2-9ada-45b1-8460-902fa2ea723c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1516 microversion: 2.1 time: 0.182297 May 24 10:14:10.911616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 50/99] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:10 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1516 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:10.924888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:10.925880 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:10.932825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:10.933536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:10.934818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:10.936119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:11.012332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lazy-loading 'flavor' on Instance uuid 75b93292-1e43-43c3-913b-eee6a6623e73 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:14:11.292573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:11.597110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:11.597800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:11.604044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:11.604493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:11.605313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:11.605754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:11.615438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-57ce11bf-bdcc-4083-8351-29cd845e01b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73/action" status: 200 len: 32 microversion: 2.1 time: 0.695921 May 24 10:14:11.617084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 50/100] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:14:10 2022] POST /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73/action => generated 32 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 24 10:14:11.638052 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:11.638971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:11.649136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:11.649625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:11.650442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:11.651055 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:11.775038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:11.789293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7c27fb64-3e67-403d-a818-56f4b15b6230 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1362 microversion: 2.1 time: 0.196981 May 24 10:14:11.801643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 51/101] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:11 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1362 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:11.810219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:11.818323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9f6c9727-df8f-4219-97ed-8c32c9c823db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.195023 May 24 10:14:11.818997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 51/102] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:11 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:12.811784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:12.812424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:12.820064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:12.820517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:12.821310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:12.821759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:12.835505 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:12.840926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:12.843292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:12.843750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:12.844679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:12.845118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:13.000740 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:13.008830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:13.009590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e54e4a62-cf54-4bf1-b00f-e45f6c61deea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.178353 May 24 10:14:13.010180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 52/103] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:12 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:13.017492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a05690c5-fe29-41b6-9352-2ae826f151b4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1548 microversion: 2.1 time: 0.209361 May 24 10:14:13.018069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 52/104] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:12 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1548 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:13.035007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:13.035497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:13.043613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:13.043613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:13.043613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:13.043613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:13.146306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-07beb205-74ac-4427-9501-beacedc0c072 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.115681 May 24 10:14:13.147540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 53/105] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:14:13 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-interface => generated 285 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:14.027127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:14.027742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:14.034621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:14.035144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:14.036123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:14.036753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:14.041041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:14.233599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:14.243067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a77b2ca8-7cd4-466b-befa-724f6e26c9e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.219084 May 24 10:14:14.243658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 53/106] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:14 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:14.700585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-1c540d25-ab40-43a5-9106-ed497c9665e8 req-d611562f-1ff3-415a-8dbe-03a1d91d7569 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ca33d7e-37bd-4d91-8a41-866155df1bc3", "tag": "697741fe-a360-4dd6-b2c9-a761d441a98d"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:14.709418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-1c540d25-ab40-43a5-9106-ed497c9665e8 req-d611562f-1ff3-415a-8dbe-03a1d91d7569 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:14.710077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-1c540d25-ab40-43a5-9106-ed497c9665e8 req-d611562f-1ff3-415a-8dbe-03a1d91d7569 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:14.725548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-1c540d25-ab40-43a5-9106-ed497c9665e8 req-d611562f-1ff3-415a-8dbe-03a1d91d7569 service nova] Creating event network-changed:697741fe-a360-4dd6-b2c9-a761d441a98d for instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 on hv-cinder-84311 May 24 10:14:14.730856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-1c540d25-ab40-43a5-9106-ed497c9665e8 req-d611562f-1ff3-415a-8dbe-03a1d91d7569 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034193 May 24 10:14:14.734404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 54/107] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:15.265208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:15.265934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:15.282907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:15.283960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:15.284913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:15.286020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:15.480589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:15.489625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ab11c6d5-a5b5-4d65-8589-79a61447ee6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.233188 May 24 10:14:15.490210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 54/108] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:15 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:16.507252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:16.508723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:16.515149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:16.515683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:16.516572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:16.517782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:16.808188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:16.816617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-096b6642-c2aa-4f7b-918f-ab780c7b342c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.314121 May 24 10:14:16.817187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 55/109] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:16 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:17.833925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:17.834570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:17.842858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:17.843315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:17.844118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:17.844559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:18.120899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:18.128999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6c161e30-fc5f-4381-85b8-b956c53d5dbd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.299583 May 24 10:14:18.129606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 55/110] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:17 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:19.146014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:19.146632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:19.153152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:19.153962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:19.154803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:19.155443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:19.326800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:19.335341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a67e023e-fab3-4654-98bb-0872b2d04dc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.193615 May 24 10:14:19.344481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 56/111] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:19 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:20.352025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:20.368039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:20.368039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:20.368039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:20.368039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:20.368533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:20.503938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:20.511340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d938ae1d-e956-492b-9a63-8a6d9256435d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.163438 May 24 10:14:20.511905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 56/112] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:20 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:21.529120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:21.529871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:21.536678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:21.537130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:21.537960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:21.538401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:21.673969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:21.681361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e8c13a7d-a1f8-487c-9980-60821ce63ee6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.156124 May 24 10:14:21.682061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 57/113] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:21 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:21.790024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:21.790531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:21.797359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:21.797810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:21.798560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:21.798992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:21.826535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:14:22.210596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e342cc2e-2769-4c5c-bcf6-90a6e2a518ae tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "DELETE /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 204 len: 0 microversion: 2.1 time: 0.424126 May 24 10:14:22.211505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 57/114] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:14:21 2022] DELETE /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:14:22.221555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:22.221555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:22.226742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:22.227193 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:22.228003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:22.228439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:22.357201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:22.365890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-776b9a62-dea3-4fb3-b099-b6ae8c25ea55 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1554 microversion: 2.1 time: 0.150076 May 24 10:14:22.366462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 58/115] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:22 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1554 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:22.700263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:22.701021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:22.707692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:22.708144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:22.708915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:22.709381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:22.888986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:22.897846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-53aa8601-4676-4606-8ccd-53221252a9c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.201646 May 24 10:14:22.898430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 58/116] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:22 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:23.382989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:23.383595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:23.389705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:23.390157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:23.390918 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:23.391357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:23.521796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:23.529903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-70691b28-c243-4256-8f10-8c2ad0c58da6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1554 microversion: 2.1 time: 0.150695 May 24 10:14:23.530483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 59/117] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:23 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1554 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:23.915334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:23.916080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:23.922566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:23.923563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:23.924573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:23.925007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:24.064500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:24.072108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f79a1a87-0eb8-4e04-a7a0-3f45b316838e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.161105 May 24 10:14:24.072677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 59/118] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:23 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:24.547004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:24.547615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:24.555659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:24.556113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:24.556883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:24.557346 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:24.866021 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:24.877077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-73caeae4-76cd-4bc2-800c-e1119052a6f5 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1554 microversion: 2.1 time: 0.334271 May 24 10:14:24.877923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 60/119] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:24 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1554 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:25.089104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:25.089747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:25.096575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:25.097105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:25.097910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:25.098341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:25.261602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:25.270440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d12e1811-a89d-4751-9491-ea9385aba2c6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.185141 May 24 10:14:25.271063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 60/120] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:25 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:25.401187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-8cb5762e-075d-4c5b-bbb1-721ecd81f652 req-bf49f302-a311-43f3-acef-db15a39ddb5f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d750c8ea-4be5-482e-957b-50352a34a21f", "name": "network-vif-deleted", "tag": "d1b0d253-6c27-4bcb-b946-972069c56a73"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:25.401187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-8cb5762e-075d-4c5b-bbb1-721ecd81f652 req-bf49f302-a311-43f3-acef-db15a39ddb5f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:25.401187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-8cb5762e-075d-4c5b-bbb1-721ecd81f652 req-bf49f302-a311-43f3-acef-db15a39ddb5f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:25.401187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-8cb5762e-075d-4c5b-bbb1-721ecd81f652 req-bf49f302-a311-43f3-acef-db15a39ddb5f service nova] Creating event network-vif-deleted:d1b0d253-6c27-4bcb-b946-972069c56a73 for instance d750c8ea-4be5-482e-957b-50352a34a21f on hv-cinder-84311 May 24 10:14:25.401556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-8cb5762e-075d-4c5b-bbb1-721ecd81f652 req-bf49f302-a311-43f3-acef-db15a39ddb5f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029902 May 24 10:14:25.401556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 61/121] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:14:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:25.895083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:25.895697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:25.902191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:25.903082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:25.904083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:25.904799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:25.986561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:25.994245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1dfd7650-47e3-4cb2-ad59-37f7980df538 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1345 microversion: 2.1 time: 0.103263 May 24 10:14:25.995369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 61/122] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:25 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1345 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:26.292078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:26.293144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:26.303582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:26.303912 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:26.312510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:26.316379 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:26.552854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:26.561811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-83c0d6a6-6c89-4f8a-9b73-1249497bd63c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.277472 May 24 10:14:26.562383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 62/123] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:26 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:27.011977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:27.012594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:27.019715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:27.020168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:27.020974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:27.021785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:27.054956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:27.056123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:27.056592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lazy-loading 'fault' on Instance uuid d750c8ea-4be5-482e-957b-50352a34a21f {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:14:27.107197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13f9b782a79f48678f297d1ef388da75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4784aebbfd1d43f896df6ca9057b3251', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:27.116691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef7e2eb3-57d7-4d7e-85c8-90c87da5675f tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 200 len: 1350 microversion: 2.1 time: 0.108154 May 24 10:14:27.125815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 62/124] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:27 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 1350 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:27.580689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:27.581670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:27.588086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:27.588547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:27.589341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:27.589814 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:27.768327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:27.776737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0a599aa0-b646-4a28-8dc9-57a720693a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.201175 May 24 10:14:27.777431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 63/125] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:27 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:28.135110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:28.135739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:28.146550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:28.148394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:28.152076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:28.152603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:28.195882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] HTTP exception thrown: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Request method failure captured: May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f HTTP/1.1 May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_Jc9L01izALm3EBVusV9cwgEeRLfcdHa-jSiI7mvThZKzDRe5stb10Uahx6MFjfriCxHekszK3A48mAWSCIzMsGV9L3qnd6ZKdYvxBzxsa_LjCtRT1T8K3o0as9iq3D9zaHXhpXEDfj9d6sARC0FU6sb4uAgL4o-tmZexZ59sm09c May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:14:28.197183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-DeleteServersTestJSON-784553224 May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 13f9b782a79f48678f297d1ef388da75 May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.198445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'd750c8ea-4be5-482e-957b-50352a34a21f'} May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:28.199194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.200517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:28.202367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Returning 404 to user: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:28.202367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-91a540e1-4948-495d-a5d6-2dce2ba7a88d tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 404 len: 111 microversion: 2.1 time: 0.068514 May 24 10:14:28.202367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 63/126] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:28 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:14:28.597320 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:28.598288 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:28.604600 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:28.605235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:28.606191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:28.606966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:28.622158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] HTTP exception thrown: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Request method failure captured: May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f/os-volume_attachments HTTP/1.1 May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_Jc9L01izALm3EBVusV9cwgEeRLfcdHa-jSiI7mvThZKzDRe5stb10Uahx6MFjfriCxHekszK3A48mAWSCIzMsGV9L3qnd6ZKdYvxBzxsa_LjCtRT1T8K3o0as9iq3D9zaHXhpXEDfj9d6sARC0FU6sb4uAgL4o-tmZexZ59sm09c May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:28.634754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-DeleteServersTestJSON-784553224 May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 13f9b782a79f48678f297d1ef388da75 May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.636644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'server_id': 'd750c8ea-4be5-482e-957b-50352a34a21f'} May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:28.640659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:28.642789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:28.643920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Returning 404 to user: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:28.643920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-337db081-43b9-40cd-a774-a0339b96db78 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.046515 May 24 10:14:28.643920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 64/127] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:14:28 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f/os-volume_attachments => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:14:28.795051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:28.796414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:28.805332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:28.806010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:28.806915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:28.807446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:28.975402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:28.986400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e8516d46-4f11-4cac-9149-8b0eea1732ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.196013 May 24 10:14:28.987028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 64/128] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:28 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:30.004868 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:30.005620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:30.012169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:30.012705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:30.013583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:30.014089 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:30.034952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:30.167277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:30.176351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23873ecd-60cb-49de-acc5-dc47c30f45ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.176458 May 24 10:14:30.176958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 65/129] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:29 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:30.230150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:30.230751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:30.233357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:30.238880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:30.239451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:30.240444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:30.241067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:30.256914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] HTTP exception thrown: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Request method failure captured: May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f HTTP/1.1 May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_Jc9L01izALm3EBVusV9cwgEeRLfcdHa-jSiI7mvThZKzDRe5stb10Uahx6MFjfriCxHekszK3A48mAWSCIzMsGV9L3qnd6ZKdYvxBzxsa_LjCtRT1T8K3o0as9iq3D9zaHXhpXEDfj9d6sARC0FU6sb4uAgL4o-tmZexZ59sm09c May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:14:30.258494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-DeleteServersTestJSON-784553224 May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 13f9b782a79f48678f297d1ef388da75 May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.259708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'd750c8ea-4be5-482e-957b-50352a34a21f'} May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:30.260346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.261456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:14:30.262600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Returning 404 to user: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:30.262600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-151d252b-9b5b-4e9d-9589-eecd2c233be6 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "DELETE /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 404 len: 111 microversion: 2.1 time: 0.033695 May 24 10:14:30.262600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 65/130] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:14:30 2022] DELETE /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:14:30.268585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:30.269203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] [instance: d750c8ea-4be5-482e-957b-50352a34a21f] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:30.271207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=72 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:30.275932 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:30.276567 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:30.277379 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:30.278004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:30.296910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] HTTP exception thrown: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Request method failure captured: May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f HTTP/1.1 May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_Jc9L01izALm3EBVusV9cwgEeRLfcdHa-jSiI7mvThZKzDRe5stb10Uahx6MFjfriCxHekszK3A48mAWSCIzMsGV9L3qnd6ZKdYvxBzxsa_LjCtRT1T8K3o0as9iq3D9zaHXhpXEDfj9d6sARC0FU6sb4uAgL4o-tmZexZ59sm09c May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:30.302733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/4784aebbfd1d43f896df6ca9057b3251"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-DeleteServersTestJSON-784553224 May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 4784aebbfd1d43f896df6ca9057b3251 May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-DeleteServersTestJSON-784553224 May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 13f9b782a79f48678f297d1ef388da75 May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-DeleteServersTestJSON-784553224-project May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.306259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': 'd750c8ea-4be5-482e-957b-50352a34a21f'} May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:30.306983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. May 24 10:14:30.308122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:30.310083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] Returning 404 to user: Instance d750c8ea-4be5-482e-957b-50352a34a21f could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:30.310083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6f20fa32-b98b-4024-8d60-bf41f9d473a4 tempest-DeleteServersTestJSON-784553224 tempest-DeleteServersTestJSON-784553224-project] 10.222.0.35 "GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f" status: 404 len: 111 microversion: 2.1 time: 0.037800 May 24 10:14:30.310083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 66/131] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:30 2022] GET /compute/v2.1/servers/d750c8ea-4be5-482e-957b-50352a34a21f => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:14:31.204323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:31.204323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:31.205445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:31.205910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:31.206705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:31.207141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:31.510528 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:31.518700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-28cb7c12-ae42-4cb2-95ce-2d314630900d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.328025 May 24 10:14:31.519259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 66/132] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:31 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:32.539570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:32.547687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:32.554181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:32.554181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:32.554661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:32.554830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:32.727303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:32.737190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ada520fd-c76f-4021-8150-d8ddfa5a6786 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.205909 May 24 10:14:32.738250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 67/133] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:32 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:33.756213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:33.757105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:33.768155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:33.768749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:33.769656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:33.770882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:33.914157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:33.922374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9f7b6df8-a3ca-41b2-8418-e2176bf42027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.171437 May 24 10:14:33.923380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 67/134] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:33 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:34.939410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:34.940040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:34.946498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:34.946946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:34.947698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:34.948130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:35.153415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:35.161733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-990c3b20-38f4-4029-abe8-ab3427174df3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.226432 May 24 10:14:35.162310 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 68/135] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:34 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:35.586307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec40e63f-54cb-4b70-9da0-c26f04dd1085 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f8732540e50>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:35.637977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec40e63f-54cb-4b70-9da0-c26f04dd1085 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/e364f2ba-2a9e-4ffb-9d73-b3863116b0f8" status: 200 len: 351 microversion: 2.1 time: 0.055441 May 24 10:14:35.638545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 68/136] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:14:35 2022] GET /compute/v2.1/os-volumes/e364f2ba-2a9e-4ffb-9d73-b3863116b0f8 => generated 351 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:36.180133 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:36.180749 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:36.186976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:36.187455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:36.188812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:36.189256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:36.360941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:36.368607 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-39febbd2-936a-4ba4-8cf1-187e0badb360 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.192763 May 24 10:14:36.369183 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 69/137] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:36 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:37.387063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:37.387781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:37.395344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:37.395822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:37.396814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:37.397247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:37.584108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:37.593409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-153a7591-228a-4c68-a640-9dc36f170871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.211158 May 24 10:14:37.593978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 69/138] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:37 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:37.733125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8e49723c-8baf-4768-b64a-fd0f6c7668b9 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f87324fa820>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:37.782099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8e49723c-8baf-4768-b64a-fd0f6c7668b9 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/47256e5f-6203-4163-9c01-93a69f55c241" status: 200 len: 350 microversion: 2.1 time: 0.052974 May 24 10:14:37.782750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 70/139] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:14:37 2022] GET /compute/v2.1/os-volumes/47256e5f-6203-4163-9c01-93a69f55c241 => generated 350 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:38.611365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:38.611983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:38.618542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:38.618992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:38.619759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:38.620197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:38.624180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:38.789197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:38.798346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-54692e68-58f8-4690-9a47-f2cdd5cac3d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.191298 May 24 10:14:38.798921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 70/140] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:38 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:39.605132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23505dae-4d56-4603-a757-d5544c7fc5db tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f8732504c10>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:39.668062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23505dae-4d56-4603-a757-d5544c7fc5db tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/3f274377-2df8-4017-864e-f09d7f779f6a" status: 200 len: 350 microversion: 2.1 time: 0.067205 May 24 10:14:39.668622 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 71/141] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:14:39 2022] GET /compute/v2.1/os-volumes/3f274377-2df8-4017-864e-f09d7f779f6a => generated 350 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:39.681028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f2fae1c4-a25a-40f6-88b8-5fadda42bba1 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f87325241f0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:39.765533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f2fae1c4-a25a-40f6-88b8-5fadda42bba1 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.088148 May 24 10:14:39.766241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 71/142] 10.222.0.35 () {58 vars in 1133 bytes} [Tue May 24 10:14:39 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:39.778118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7197d60f-4947-40af-9c6b-84095c5d2dd4 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f87324ba0d0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:39.818204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:39.821934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:39.839782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:39.840473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:39.843735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:39.844571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:39.883490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7197d60f-4947-40af-9c6b-84095c5d2dd4 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.108978 May 24 10:14:39.891265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 72/143] 10.222.0.35 () {58 vars in 1148 bytes} [Tue May 24 10:14:39 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:39.896754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7333bff4-4c78-4fe3-9757-2f525e42896b tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f8732504670>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:39.991801 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7333bff4-4c78-4fe3-9757-2f525e42896b tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.098696 May 24 10:14:39.992370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 73/144] 10.222.0.35 () {58 vars in 1133 bytes} [Tue May 24 10:14:39 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:40.004473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2af514a-f67f-4de6-a9aa-93db2761b8f6 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f87324fadc0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:40.005247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:40.026803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-987314a9-4d21-4cf0-ad31-009605e5a6e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.214348 May 24 10:14:40.027716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 72/145] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:39 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:40.088900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2af514a-f67f-4de6-a9aa-93db2761b8f6 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.088928 May 24 10:14:40.089610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 74/146] 10.222.0.35 () {58 vars in 1166 bytes} [Tue May 24 10:14:39 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:14:40.101560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3d25674a-d97d-406d-855e-4365cbf012c0 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f8732524160>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:40.192582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3d25674a-d97d-406d-855e-4365cbf012c0 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.094738 May 24 10:14:40.193192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 73/147] 10.222.0.35 () {58 vars in 1169 bytes} [Tue May 24 10:14:40 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:40.205777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fdc0f8cb-6b0e-4766-a0de-c5aaad12aef2 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f87324ba5e0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:40.290923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fdc0f8cb-6b0e-4766-a0de-c5aaad12aef2 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.089211 May 24 10:14:40.291488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 75/148] 10.222.0.35 () {58 vars in 1154 bytes} [Tue May 24 10:14:40 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:40.303444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6f4919a7-a668-42e6-8e91-1095cf3b6ce3 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f8732540550>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:40.384887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6f4919a7-a668-42e6-8e91-1095cf3b6ce3 tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.085014 May 24 10:14:40.385554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 74/149] 10.222.0.35 () {58 vars in 1187 bytes} [Tue May 24 10:14:40 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:14:40.397679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0f1e2c03-f572-40b8-9178-5766bcfeaccb tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] Calling method '.version_select at 0x7f87324fa430>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:40.481405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0f1e2c03-f572-40b8-9178-5766bcfeaccb tempest-VolumesTestJSON-1298158274 tempest-VolumesTestJSON-1298158274-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.087516 May 24 10:14:40.482172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 76/150] 10.222.0.35 () {58 vars in 1154 bytes} [Tue May 24 10:14:40 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:41.044651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:41.045195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:41.052546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:41.053006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:41.053805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:41.054240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:41.239816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:41.256408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6445ffd3-d75b-4856-8359-64b8629290dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.216243 May 24 10:14:41.258715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 75/151] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:41 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:42.276418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:42.277485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:42.284775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:42.285471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:42.286846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:42.287522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:42.458767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:42.470503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5fa33024-64d9-4d5c-ac53-d14dcc3b4536 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.198432 May 24 10:14:42.471404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 77/152] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:42 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:43.488309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:43.488861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:43.495932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:43.496560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:43.497379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:43.497804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:43.637402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:43.655133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4e06aec4-ff68-4ce5-aebe-0bbcae9d0744 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.172107 May 24 10:14:43.655841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 76/153] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:43 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:44.674429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:44.675099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:44.682277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:44.682483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:44.683200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:44.683708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:44.832850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:44.841838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4d10fe3e-12f7-4bfa-aff3-c162825edf85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.171916 May 24 10:14:44.853230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 78/154] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:44 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:45.178203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-890526005", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "networks": [{"uuid": "8c0e95bb-fac1-46f0-9283-fbcc016bef68"}]}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:45.862109 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:45.862715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:45.873028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:45.873854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:45.876444 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:45.876444 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:46.075240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:46.084032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6d9fde36-7088-44ae-9664-abafadf7b921 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.226283 May 24 10:14:46.084602 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 79/155] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:45 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:47.102601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:47.104049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:47.111817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:47.112271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:47.113035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:47.113603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:47.286813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:47.294261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-73f57aa4-c3a5-4eb8-b50d-ebcb67c0a512 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1522 microversion: 2.1 time: 0.195809 May 24 10:14:47.294825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 80/156] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:47 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:14:48.065807 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-66af47bd-930e-490d-ae1b-cd6b993e7dae tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Action: 'create', calling method: .version_select at 0x7f87326b1ee0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1920827983", "metadata": {"Type": "work"}}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-66af47bd-930e-490d-ae1b-cd6b993e7dae tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Request method failure captured: May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 122 May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLAXSbk4vMB_oYuYBN9MD9z82b76nrTHcvl6eKY4myTsp46sJqOC3VAYYJBN6t6Bfj85EAo4dCJn0hNl-Fa5Ejd7LznqPzRIT6tny1oi0BOVt-pFBFzjq4KI0l4M_e5MtIyd5w4T-D38wog8z9_EGUnMU2885HUErWIBXDWH7BfZWzU May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:48.080343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: ea54d36f1dc24c38ae7c7ef34072cd83 May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1920827983", "metadata": {"Type": "work"}}} May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:14:48.081897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1920827983', 'metadata': {'Type': 'work'}}}} May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise error May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi On instance['volume']: May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi '#$%' May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:48.082771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 24 10:14:48.084053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 24 10:14:48.085348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' May 24 10:14:48.085348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.088039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-66af47bd-930e-490d-ae1b-cd6b993e7dae tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:48.089003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-66af47bd-930e-490d-ae1b-cd6b993e7dae tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.154108 May 24 10:14:48.089862 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 81/157] 10.222.0.35 () {60 vars in 1155 bytes} [Tue May 24 10:14:47 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 155 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 24 10:14:48.099674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2522aaa6-3628-4b41-b949-3e308884d6b8 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Action: 'create', calling method: .version_select at 0x7f87324ba5e0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1861661053", "metadata": {"Type": "work"}}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2522aaa6-3628-4b41-b949-3e308884d6b8 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Request method failure captured: May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 120 May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLAXSbk4vMB_oYuYBN9MD9z82b76nrTHcvl6eKY4myTsp46sJqOC3VAYYJBN6t6Bfj85EAo4dCJn0hNl-Fa5Ejd7LznqPzRIT6tny1oi0BOVt-pFBFzjq4KI0l4M_e5MtIyd5w4T-D38wog8z9_EGUnMU2885HUErWIBXDWH7BfZWzU May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:48.102219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: ea54d36f1dc24c38ae7c7ef34072cd83 May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1861661053", "metadata": {"Type": "work"}}} May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:14:48.104214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1861661053', 'metadata': {'Type': 'work'}}}} May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise error May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi On instance['volume']: May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi '0' May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:48.105054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 24 10:14:48.106608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 24 10:14:48.107952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 May 24 10:14:48.107952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.107952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2522aaa6-3628-4b41-b949-3e308884d6b8 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:48.107952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2522aaa6-3628-4b41-b949-3e308884d6b8 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.008317 May 24 10:14:48.108571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 82/158] 10.222.0.35 () {60 vars in 1155 bytes} [Tue May 24 10:14:48 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 24 10:14:48.114467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b48c8c42-6eeb-4cb8-9dd0-4fe008520306 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Action: 'create', calling method: .version_select at 0x7f8732751280>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1747214912", "metadata": {"Type": "work"}}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b48c8c42-6eeb-4cb8-9dd0-4fe008520306 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Request method failure captured: May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: POST /compute/v2.1/os-volumes HTTP/1.1 May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 119 May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLAXSbk4vMB_oYuYBN9MD9z82b76nrTHcvl6eKY4myTsp46sJqOC3VAYYJBN6t6Bfj85EAo4dCJn0hNl-Fa5Ejd7LznqPzRIT6tny1oi0BOVt-pFBFzjq4KI0l4M_e5MtIyd5w4T-D38wog8z9_EGUnMU2885HUErWIBXDWH7BfZWzU May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:48.116753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: ea54d36f1dc24c38ae7c7ef34072cd83 May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1747214912", "metadata": {"Type": "work"}}} May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:14:48.118155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1747214912', 'metadata': {'Type': 'work'}}}} May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise error May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi On instance['volume']: May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi '' May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:48.119026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ May 24 10:14:48.120529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( May 24 10:14:48.121718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' May 24 10:14:48.121718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.121718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b48c8c42-6eeb-4cb8-9dd0-4fe008520306 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:48.121718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b48c8c42-6eeb-4cb8-9dd0-4fe008520306 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.008568 May 24 10:14:48.121718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 83/159] 10.222.0.35 () {60 vars in 1155 bytes} [Tue May 24 10:14:48 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 24 10:14:48.127563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3597afda-2c67-4084-a48f-af5581b4c8eb tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Calling method '.version_select at 0x7f87324fa940>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:48.157523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-3597afda-2c67-4084-a48f-af5581b4c8eb tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] HTTP exception thrown: Volume tempest-invalid-528961042 could not be found. May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3597afda-2c67-4084-a48f-af5581b4c8eb tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Request method failure captured: May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-528961042 HTTP/1.1 May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLAXSbk4vMB_oYuYBN9MD9z82b76nrTHcvl6eKY4myTsp46sJqOC3VAYYJBN6t6Bfj85EAo4dCJn0hNl-Fa5Ejd7LznqPzRIT6tny1oi0BOVt-pFBFzjq4KI0l4M_e5MtIyd5w4T-D38wog8z9_EGUnMU2885HUErWIBXDWH7BfZWzU May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:48.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: ea54d36f1dc24c38ae7c7ef34072cd83 May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume tempest-invalid-528961042 could not be found. May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': 'tempest-invalid-528961042'} May 24 10:14:48.161315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-528961042 could not be found. (HTTP 404) (Request-ID: req-a400722a-4814-4c08-9755-f152a8025c57) May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.162171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 24 10:14:48.163641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-528961042 could not be found. May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:48.164857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-528961042 could not be found. May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-528961042 could not be found. May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3597afda-2c67-4084-a48f-af5581b4c8eb tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Returning 404 to user: Volume tempest-invalid-528961042 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3597afda-2c67-4084-a48f-af5581b4c8eb tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/tempest-invalid-528961042" status: 404 len: 98 microversion: 2.1 time: 0.037991 May 24 10:14:48.170856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 84/160] 10.222.0.35 () {58 vars in 1214 bytes} [Tue May 24 10:14:48 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-528961042 => generated 98 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) May 24 10:14:48.172780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-62a2f150-78e0-4473-9ab4-b82a07f4582e tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004931 May 24 10:14:48.173315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 85/161] 10.222.0.35 () {58 vars in 1139 bytes} [Tue May 24 10:14:48 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) May 24 10:14:48.181298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-05178496-8426-4238-9900-231bb04c2631 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003347 May 24 10:14:48.182063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 86/162] 10.222.0.35 () {58 vars in 1136 bytes} [Tue May 24 10:14:48 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 4 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) May 24 10:14:48.190757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f5ce021b-371d-4fe7-9ec4-0e954d50cccd tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Calling method '.version_select at 0x7f87324fa4c0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:48.222222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-f5ce021b-371d-4fe7-9ec4-0e954d50cccd tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] HTTP exception thrown: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f5ce021b-371d-4fe7-9ec4-0e954d50cccd tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Request method failure captured: May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: DELETE /compute/v2.1/os-volumes/02e5c32e-0362-4659-a7b8-cafccc7ae701 HTTP/1.1 May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLAXSbk4vMB_oYuYBN9MD9z82b76nrTHcvl6eKY4myTsp46sJqOC3VAYYJBN6t6Bfj85EAo4dCJn0hNl-Fa5Ejd7LznqPzRIT6tny1oi0BOVt-pFBFzjq4KI0l4M_e5MtIyd5w4T-D38wog8z9_EGUnMU2885HUErWIBXDWH7BfZWzU May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:48.224296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: ea54d36f1dc24c38ae7c7ef34072cd83 May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '02e5c32e-0362-4659-a7b8-cafccc7ae701'} May 24 10:14:48.226108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. (HTTP 404) (Request-ID: req-6995b760-3099-4557-9aac-4fd1104b032d) May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.231565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._delete(loc) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete May 24 10:14:48.232792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete May 24 10:14:48.234130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f5ce021b-371d-4fe7-9ec4-0e954d50cccd tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Returning 404 to user: Volume 02e5c32e-0362-4659-a7b8-cafccc7ae701 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f5ce021b-371d-4fe7-9ec4-0e954d50cccd tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "DELETE /compute/v2.1/os-volumes/02e5c32e-0362-4659-a7b8-cafccc7ae701" status: 404 len: 109 microversion: 2.1 time: 0.038946 May 24 10:14:48.236041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 87/163] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:14:48 2022] DELETE /compute/v2.1/os-volumes/02e5c32e-0362-4659-a7b8-cafccc7ae701 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:14:48.236950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-be56e3b5-9d36-4478-b8be-eee4f460bd82 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Calling method '.version_select at 0x7f8732504af0>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:48.264149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-be56e3b5-9d36-4478-b8be-eee4f460bd82 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] HTTP exception thrown: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-be56e3b5-9d36-4478-b8be-eee4f460bd82 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Request method failure captured: May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/os-volumes/bd8a2efa-3d84-477e-9773-768c7789e6a7 HTTP/1.1 May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLAXSbk4vMB_oYuYBN9MD9z82b76nrTHcvl6eKY4myTsp46sJqOC3VAYYJBN6t6Bfj85EAo4dCJn0hNl-Fa5Ejd7LznqPzRIT6tny1oi0BOVt-pFBFzjq4KI0l4M_e5MtIyd5w4T-D38wog8z9_EGUnMU2885HUErWIBXDWH7BfZWzU May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:14:48.266909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c2ec636b05634843b9f8eadc5ae82b1d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: c2ec636b05634843b9f8eadc5ae82b1d May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesNegativeTest-1688474913 May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: ea54d36f1dc24c38ae7c7ef34072cd83 May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesNegativeTest-1688474913-project May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': 'bd8a2efa-3d84-477e-9773-768c7789e6a7'} May 24 10:14:48.268272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. (HTTP 404) (Request-ID: req-3b63d5ee-6cda-471e-ac85-5502341b5f76) May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.269294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi item = cinderclient( May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get May 24 10:14:48.270591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show May 24 10:14:48.271601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-be56e3b5-9d36-4478-b8be-eee4f460bd82 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] Returning 404 to user: Volume bd8a2efa-3d84-477e-9773-768c7789e6a7 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-be56e3b5-9d36-4478-b8be-eee4f460bd82 tempest-VolumesNegativeTest-1688474913 tempest-VolumesNegativeTest-1688474913-project] 10.222.0.35 "GET /compute/v2.1/os-volumes/bd8a2efa-3d84-477e-9773-768c7789e6a7" status: 404 len: 109 microversion: 2.1 time: 0.036551 May 24 10:14:48.272596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 88/164] 10.222.0.35 () {58 vars in 1244 bytes} [Tue May 24 10:14:48 2022] GET /compute/v2.1/os-volumes/bd8a2efa-3d84-477e-9773-768c7789e6a7 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:14:48.310838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:48.311526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:48.318221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:48.318674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:48.319440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:48.319877 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:48.455109 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:48.462662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1c860b31-8e45-4ed0-82ff-639770e905ae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1502 microversion: 2.1 time: 0.155742 May 24 10:14:48.463233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 89/165] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:48 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1502 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:48.479983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:48.480623 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:48.486943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:48.487400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:48.488167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:48.488603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:48.629823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:48.637339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e4b638a1-ed5b-4523-a178-a421ee29693b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1517 microversion: 2.1 time: 0.161241 May 24 10:14:48.637903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 90/166] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:48 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1517 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:50.641668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:14:50.643311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:50.649381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:50.650025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:50.651048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:50.651495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:50.679017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lazy-loading 'flavor' on Instance uuid b3484367-cf43-4128-b7f2-96124543234c {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:14:51.022698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1fe8377f-5915-4574-b986-ac07fef01ca1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/action" status: 200 len: 32 microversion: 2.1 time: 0.385067 May 24 10:14:51.023309 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 91/167] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:14:50 2022] POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/action => generated 32 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 24 10:14:51.033799 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:51.034498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:51.041981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:51.042436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:51.043203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:51.043639 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:51.208991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:51.231941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e78beefe-162a-4853-834c-5632e2b140e1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.193822 May 24 10:14:51.231941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 92/168] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:51 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:52.240688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:52.241357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:52.247334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:52.247782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:52.248545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:52.248989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:52.397534 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:52.405443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4cd3c737-5de9-4f00-9465-c6f97bb4d890 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.168652 May 24 10:14:52.406208 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 93/169] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:52 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:53.424623 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:53.425244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:53.431594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:53.432048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:53.432821 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:53.433280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:53.567761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:53.580113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7a99d874-2e23-450d-a4fd-9ead853e53c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.160224 May 24 10:14:53.580714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 94/170] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:53 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:53.641510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=23 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:54.597636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:54.599002 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:54.605483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:54.605837 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:54.606594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:54.607024 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:54.763388 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:54.767792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:54.779655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-340ca710-07d2-4615-8f41-3371153d7589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.186437 May 24 10:14:54.780437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 95/171] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:54 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:55.187974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=17 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:55.800571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:55.801733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:55.813815 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:55.815483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:55.816699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:55.817306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:55.951968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:55.959488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1cee5b78-19e0-4cf8-84aa-2d38f467fe63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.165522 May 24 10:14:55.960062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 96/172] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:55 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:56.976674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:56.977326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:56.983965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:56.985127 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:56.985943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:56.986368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:57.119754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:57.127819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2b705b04-664f-47e9-837e-c86bcf375cce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.154896 May 24 10:14:57.128424 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 97/173] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:56 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:58.145823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:58.148252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:58.155617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:58.156272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:58.157216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:58.157989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:58.329929 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:58.339431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c54d5400-dcf8-4eeb-891f-3fa75b2c9105 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.198587 May 24 10:14:58.340262 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 98/174] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:58 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:14:58.405050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:58.405573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:58.415058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:58.415058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:58.415534 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:58.415987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:58.444907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4cf87279-7feb-4b66-a996-4468769003dd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/ips" status: 200 len: 145 microversion: 2.1 time: 0.043775 May 24 10:14:58.445574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 99/175] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:14:58 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/ips => generated 145 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:14:59.357294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:14:59.357964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:14:59.364705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:59.365158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:59.365960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:14:59.366452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:14:59.509108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:14:59.518866 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c73960df-4f52-4db9-924a-334033176339 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.165520 May 24 10:14:59.519463 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 100/176] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:14:59 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:00.537076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:00.537827 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:00.543708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:00.544154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:00.544904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:00.545368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:00.714381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:00.726281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1d180798-468f-492c-a5ab-300a7c478b2f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.193038 May 24 10:15:00.726869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 101/177] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:00 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:01.743090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:01.743684 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:01.750784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:01.751229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:01.753778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:01.753778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:01.903111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:01.912274 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2cfdfb15-3dc4-4d86-9c58-05041304105b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.173032 May 24 10:15:01.912875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 102/178] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:01 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:01.917596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a1037a38-e9c8-4a1f-8c9f-b661216734dc"}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:01.921818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:01.924750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:01.925201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:01.925991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:01.926429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:02.143402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] validate_networks() for [('8c0e95bb-fac1-46f0-9283-fbcc016bef68', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:15:02.521228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:15:02.521690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:15:02.522162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:15:02.522602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:15:02.523008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:15:02.532540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:02.532707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:02.533592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:02.534027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:02.552550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Getting quotas for project a805d83a8e7747528285d543eb274ab6. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:15:02.561374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Getting quotas for user 6e8f1cb9760f41c6b7c1eb379daf1b64 and project a805d83a8e7747528285d543eb274ab6. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:15:02.573843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:15:02.576196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:15:02.735810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-12b96212-f45e-4342-8f83-96d64b46ac64 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 17.562294 May 24 10:15:02.736739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 77/179] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:14:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 17565 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:15:02.746852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e9ed8359-7f0a-4ecf-8575-32bff2cf6e8c tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:02.747485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e9ed8359-7f0a-4ecf-8575-32bff2cf6e8c tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:02.820428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e9ed8359-7f0a-4ecf-8575-32bff2cf6e8c tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:02.830364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e9ed8359-7f0a-4ecf-8575-32bff2cf6e8c tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1213 microversion: 2.1 time: 0.087300 May 24 10:15:02.831458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 78/180] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:02 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1213 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:02.930101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:02.930705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:02.938268 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:02.943241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:02.943241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:02.943241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:03.047520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-acbe9944-cb3b-41a5-9675-6e7df1dce704 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.132600 May 24 10:15:03.048857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 103/181] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:15:01 2022] POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-volume_attachments => generated 194 bytes in 1135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:03.068523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:03.076054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aaceaecb-ffdd-44cc-88a6-de44011dc730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.150011 May 24 10:15:03.076629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 79/182] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:02 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:03.848858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-81f1ba16-d867-4a2a-8bea-103a8892e6de tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:03.849671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-81f1ba16-d867-4a2a-8bea-103a8892e6de tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:03.851684 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=46 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:03.904706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-81f1ba16-d867-4a2a-8bea-103a8892e6de tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:03.911471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-81f1ba16-d867-4a2a-8bea-103a8892e6de tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1213 microversion: 2.1 time: 0.067639 May 24 10:15:03.912038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 104/183] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:03 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1213 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:04.092959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:04.094125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:04.100674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:04.101062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:04.101852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:04.102286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:04.250073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:04.257985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9b53c211-f0ba-4197-87a4-53cabf56ae34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.168848 May 24 10:15:04.265838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 80/184] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:04 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:04.929581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:04.930327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:04.938786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:04.939174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:04.939991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:04.940439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:05.008562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:05.016742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fadc89d6-d0e8-416c-a3b5-86c674310349 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1209 microversion: 2.1 time: 0.091624 May 24 10:15:05.017382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 105/185] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:04 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1209 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:05.275405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:05.276024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:05.283562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:05.284018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:05.284793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:05.285429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:05.424883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:05.432425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-976485a2-100e-4b52-b111-fd34bb279643 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.160961 May 24 10:15:05.432998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 81/186] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:05 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:06.034756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:06.036144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:06.050116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:06.050747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:06.051913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:06.052519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:06.133166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:06.141356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-924b8ba2-2017-4d0c-b6f9-c57f5ad31b2e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1283 microversion: 2.1 time: 0.110805 May 24 10:15:06.142146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 106/187] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:06 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1283 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:06.451115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:06.452472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:06.461005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:06.461742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:06.462925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:06.463517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:06.649921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:06.677911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-68811d28-bf9e-4e81-be3a-e5501a62e2b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.227205 May 24 10:15:06.678549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 82/188] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:06 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:07.162222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:07.164168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:07.194569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:07.195364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:07.196662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:07.197628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:07.430669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:07.439689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e74c0840-1d3c-4729-bb80-f8abc22b1b01 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.284374 May 24 10:15:07.440596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 107/189] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:07 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:07.695639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:07.696241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:07.703728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:07.704177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:07.704947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:07.705425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:07.891814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:07.901385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-707c2195-749e-4300-b79a-3afb6ea303d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.209661 May 24 10:15:07.902784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 83/190] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:07 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:08.022761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-026fcf41-869b-49d1-9caa-eadd4ab4d49e req-9634e846-be29-4b8c-88d5-6d648ed46e36 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7bb62cee-05f2-450d-96e7-ad37469c13e5", "tag": "b9988d12-0125-4f5b-9219-74dda5e4ee6c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:08.040313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-026fcf41-869b-49d1-9caa-eadd4ab4d49e req-9634e846-be29-4b8c-88d5-6d648ed46e36 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:08.041013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-026fcf41-869b-49d1-9caa-eadd4ab4d49e req-9634e846-be29-4b8c-88d5-6d648ed46e36 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:08.055997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-026fcf41-869b-49d1-9caa-eadd4ab4d49e req-9634e846-be29-4b8c-88d5-6d648ed46e36 service nova] Creating event network-changed:b9988d12-0125-4f5b-9219-74dda5e4ee6c for instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 on hv-cinder-84311 May 24 10:15:08.061078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-026fcf41-869b-49d1-9caa-eadd4ab4d49e req-9634e846-be29-4b8c-88d5-6d648ed46e36 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046934 May 24 10:15:08.062128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 108/191] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:15:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:08.458718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:08.458935 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:08.467633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:08.469108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:08.470276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:08.472884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:08.668559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:08.677689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-22826d58-dacf-4805-85b6-4da29fff216b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.224897 May 24 10:15:08.678276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 84/192] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:08 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:08.923724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:08.924889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:08.933677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:08.934190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:08.935123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:08.935774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:09.085669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:09.093777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-be800a76-368f-48fb-9c30-f313a39ec6a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.176505 May 24 10:15:09.094506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 109/193] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:08 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:09.694781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:09.695410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:09.703325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:09.703810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:09.704785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:09.705220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:09.874065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:09.888050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0c3f493a-1364-445f-bc8e-f5c36dfde894 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.193551 May 24 10:15:09.894735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 85/194] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:09 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:10.111460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:10.112069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:10.121051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:10.121875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:10.122671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:10.123114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:10.362609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:10.376356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5a58fe5a-906b-4ab7-845a-d3ed0f72c555 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.268907 May 24 10:15:10.384546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 110/195] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:10 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:10.903265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:10.904553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:10.914147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:10.915211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:10.916048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:10.917210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:11.236022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:11.244453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ab023bfa-ed82-48c8-b6e0-91079bdef9a4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.346464 May 24 10:15:11.247306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 86/196] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:10 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:11.396782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:11.397384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:11.406435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:11.407295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:11.408558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:11.418222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:11.611110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:11.619807 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b4bbd4c-0a6e-40ff-9913-8febfaf70732 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.229826 May 24 10:15:11.620711 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 111/197] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:11 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:12.098244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b895c2e5-5161-4348-aac9-d16521800f0f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:12.110512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b895c2e5-5161-4348-aac9-d16521800f0f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.112509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b895c2e5-5161-4348-aac9-d16521800f0f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.212317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b895c2e5-5161-4348-aac9-d16521800f0f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Going to try to stop instance {{(pid=118232) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 24 10:15:12.263050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:12.264142 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:12.275172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.275909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.276694 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.277478 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.462081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:12.471432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-de627669-04af-42c8-b17e-66dd589ce10b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.212310 May 24 10:15:12.482318 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 112/198] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:12 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:12.503325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b895c2e5-5161-4348-aac9-d16521800f0f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.408970 May 24 10:15:12.505176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 87/199] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:15:12 2022] POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action => generated 0 bytes in 411 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:15:12.512716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:12.513388 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:12.520280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.520737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.521638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.522079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.526307 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:12.639395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:12.643021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:12.643760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:12.655579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.656547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.657356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:12.657805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:12.691685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:12.704071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-27915914-fcc5-4a81-9651-5182aa8d82eb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.194873 May 24 10:15:12.704752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 113/200] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:12 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:12.825985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=8 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:12.879760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:12.887914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-86ef198c-caa2-45c6-9187-d6e3c5de685d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.251503 May 24 10:15:12.890396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 88/201] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:12 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:13.488629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:13.489383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:13.498122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:13.498812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:13.500499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:13.500943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:13.693799 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:13.704184 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5a7b8c0a-3d82-4d02-a424-0684ac24f839 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.218106 May 24 10:15:13.705116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 114/202] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:13 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:13.723773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:13.724614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:13.732961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:13.733550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:13.734406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:13.735057 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:13.909135 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:13.909698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:13.914418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:13.930155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:13.947027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:13.947027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:13.947027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:14.070444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:14.081234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-24871886-39f8-424b-8e43-4da996048d58 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.361217 May 24 10:15:14.082450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 89/203] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:13 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:14.107536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:14.116072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-920786d6-4749-4f0f-a233-4209ff6abd1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.212740 May 24 10:15:14.117095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 115/204] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:13 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:14.722036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:14.722714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:14.733301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:14.734230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:14.734596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:14.735219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:15.030080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:15.067285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a9a7f07-cd96-43a7-b255-b9f3707a9bee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.349560 May 24 10:15:15.097851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 90/205] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:14 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:15.100276 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:15.101012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:15.109238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:15.110832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:15.115122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:15.115938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:15.134832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:15.135366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:15.143246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:15.144023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:15.144969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:15.145606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:15.326901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:15.336070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-82a42072-aa89-4bc9-a8db-75117c4496a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.239902 May 24 10:15:15.346809 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 116/206] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:15 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:15.442922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:15.453344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fcb135ba-d276-451e-b8cb-c57783829fad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.321486 May 24 10:15:15.454371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 91/207] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:15 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:16.099277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:16.102255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:16.107771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.108561 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.110114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.110702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.370537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:16.371455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:16.397230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.400836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.401839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.402989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.681083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:16.699844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a9b82a6c-eaf6-44b7-8e14-62d577878483 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.343056 May 24 10:15:16.702046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 92/208] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:16 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:16.717105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:16.718052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:16.731647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.732316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.735018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.735487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.752767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:16.762912 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e706c24d-c1dd-4456-a6b5-300098063783 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.670528 May 24 10:15:16.772961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 117/209] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:16 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:16.898872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:16.915375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-08ac6f82-b32e-4c7e-8da1-b3771c40a5fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.211087 May 24 10:15:16.915944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 93/210] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:16 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:16.941443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-4915a851-21db-4e79-b672-9f70d03d3533 req-293d7e3b-1505-4d6f-a554-a2db1c9bd2e1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7bb62cee-05f2-450d-96e7-ad37469c13e5", "name": "network-vif-plugged", "status": "completed", "tag": "b9988d12-0125-4f5b-9219-74dda5e4ee6c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:16.951157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-4915a851-21db-4e79-b672-9f70d03d3533 req-293d7e3b-1505-4d6f-a554-a2db1c9bd2e1 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:16.951646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-4915a851-21db-4e79-b672-9f70d03d3533 req-293d7e3b-1505-4d6f-a554-a2db1c9bd2e1 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:16.963802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-4915a851-21db-4e79-b672-9f70d03d3533 req-293d7e3b-1505-4d6f-a554-a2db1c9bd2e1 service nova] Creating event network-vif-plugged:b9988d12-0125-4f5b-9219-74dda5e4ee6c for instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 on hv-cinder-84311 May 24 10:15:16.969808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-4915a851-21db-4e79-b672-9f70d03d3533 req-293d7e3b-1505-4d6f-a554-a2db1c9bd2e1 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035625 May 24 10:15:16.970597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 118/211] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:15:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:17.721040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:17.722242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:17.734050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:17.734050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:17.734050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:17.734050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:17.738174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:17.782583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:17.784033 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:17.791804 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:17.792259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:17.793207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:17.793956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:17.899942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:17.909330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-55d52f16-5d42-43f7-91bc-7fc808dbd563 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.192796 May 24 10:15:17.919680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 94/212] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:17 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:17.939189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:17.940027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:17.947405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:17.947911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:17.948746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:17.949383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:18.036229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:18.062420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23f627a7-282a-4658-90cc-390ab6cdf006 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.282573 May 24 10:15:18.065462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 119/213] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:17 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:18.195641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:18.206931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aa9ad240-477c-40e9-bbc5-8975ad25fcb4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.276854 May 24 10:15:18.207718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 95/214] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:17 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:18.929020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:18.930682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:18.941336 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:18.942756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:18.943869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:18.944496 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:19.080823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:19.081452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:19.091833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:19.098814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:19.101592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8c9a135e-c550-4e03-b023-0b1eb8b0b5ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.176958 May 24 10:15:19.103123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:19.107609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:19.108226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:19.112165 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 120/215] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:18 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:19.230439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:19.231087 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:19.238515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:19.238984 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:19.239938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:19.240372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:19.352420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:19.361753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e77ce6dc-982d-4bc3-9077-c5afdcf8ae1f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.285170 May 24 10:15:19.362544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 96/216] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:19 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:19.411190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:19.425671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a7f33f2e-501f-4765-9428-30ae23620559 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.203331 May 24 10:15:19.430532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 121/217] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:19 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:20.120563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:20.121169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:20.128964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:20.129441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:20.130213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:20.130637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:20.338464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:20.347724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b2dde3b6-0df3-4f38-adc8-8e35d84138c4 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.230989 May 24 10:15:20.349035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 97/218] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:20 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:20.379843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:20.380463 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:20.387316 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:20.387775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:20.388547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:20.389337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:20.453870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:20.458293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:20.471088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:20.483777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:20.484955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:20.485648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:20.706091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:20.726709 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cd8bc918-9ee3-4867-b404-607799a8fb04 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.350487 May 24 10:15:20.728073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 122/219] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:20 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:20.743382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:20.753462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eedc2844-0d56-4b5e-b2cd-411d74713c8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.308664 May 24 10:15:20.754753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 98/220] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:20 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:21.365786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:21.366676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:21.375871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:21.378487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:21.379506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:21.379952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:21.660114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:21.668632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-afa614e1-6d9e-4f8d-897c-00f87e0215e2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.306634 May 24 10:15:21.672719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 123/221] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:21 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:21.752998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:21.754401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:21.769235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:21.769975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:21.771163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:21.771525 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:21.772242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:21.774700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:21.783120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:21.783647 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:21.784611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:21.785211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:21.965819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:21.974944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4ec37a5f-a639-4f39-9c58-629522e81cdc tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1313 microversion: 2.1 time: 0.234611 May 24 10:15:21.976126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 99/222] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:21 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1313 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:22.032221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:22.041147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7d844466-30a9-41ff-bb6c-3dedb96f0a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.273955 May 24 10:15:22.041773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 124/223] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:21 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:22.689763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:22.690373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:22.731712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:22.731712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:22.731712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:22.731712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:22.932588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:22.941905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4aa7d644-c09e-48b0-8e67-0ea03dece644 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.256856 May 24 10:15:22.942962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 100/224] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:22 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:22.992533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:22.993323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:23.001437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:23.002080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:23.003008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:23.004111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:23.059584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:23.060204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:23.066664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:23.067119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:23.068037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:23.068693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:23.182233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:23.190990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-650940b3-a7b6-44f2-b54e-bce35121f591 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1496 microversion: 2.1 time: 0.202508 May 24 10:15:23.210965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 125/225] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:22 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1496 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:23.364712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:23.373350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a8a83b4a-f0df-4317-a793-d951a36b6834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.318979 May 24 10:15:23.374106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 101/226] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:23 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:23.961440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:23.962061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:23.970216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:23.970858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:23.972507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:23.972507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:24.198415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:24.208980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aa67db41-fb80-4195-bd8c-7663439209ed tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.251245 May 24 10:15:24.209789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 126/227] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:23 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:24.390515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:24.391181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:24.407556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:24.408108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:24.410620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:24.411066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:24.911712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:24.921825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-74c0895b-2af4-4d5e-b786-21fd5c43c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.534209 May 24 10:15:24.922750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 102/228] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:24 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:25.230050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:25.230723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:25.237868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:25.238387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:25.239188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:25.239611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:25.325949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-15191f71-bdac-4138-b76a-1b194ce9d0fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Action: 'create', calling method: .version_select at 0x7f8732504430>, body: {"keypair": {"name": "tempest-TestShelveInstance-91633904", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEQrYQXQW6wxjpbxlDz0t5yU5gBGbqwRP9Y2mAFTjvDQoKFF3YjdCaxK8Upp+W4TDIhLRb7F2Zic5E7xU82TP9oNs7+70a7V/FYIJVDomUBynIVux9leAveZC2rvZNauPw=="}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:25.340806 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-15191f71-bdac-4138-b76a-1b194ce9d0fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Getting quotas for project 5bf4c198860648cd874233ba27d1b971. Resources: {'key_pairs'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:15:25.345581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-15191f71-bdac-4138-b76a-1b194ce9d0fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Getting quotas for user 577c7840904d4f2180f7e5bd0f530a9a and project 5bf4c198860648cd874233ba27d1b971. Resources: {'key_pairs'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:15:25.377671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:25.385654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a5a3fd1c-d803-4772-9a33-4226e40043c9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.159668 May 24 10:15:25.386227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 103/229] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:25 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:25.466275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-15191f71-bdac-4138-b76a-1b194ce9d0fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 395 microversion: 2.1 time: 0.314357 May 24 10:15:25.466885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 127/230] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:15:25 2022] POST /compute/v2.1/os-keypairs => generated 395 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:25.939074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:25.939686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:25.942094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:25.946936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:25.947386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:25.948157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:25.948596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:26.120111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:26.123227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5d4b00d5-f92b-40ab-b87b-32383a076952 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.188128 May 24 10:15:26.124186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 104/231] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:25 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:26.407182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:26.407182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:26.414405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:26.415125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:26.416087 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:26.416842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:26.598420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:26.606212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-27d212ad-f37d-49ea-be35-7450492bdaf9 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.206657 May 24 10:15:26.606899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 128/232] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:26 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:27.143079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:27.143891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:27.153956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:27.154447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:27.155417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:27.156233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:27.307846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:27.321565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9950fe48-2e23-45f6-8b27-524e5dbcde0a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.179454 May 24 10:15:27.321565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 105/233] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:27 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:27.624217 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:27.625949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:27.635327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:27.635958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:27.637067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:27.638106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:27.822125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:27.834445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d4260eff-db24-4925-812c-5b5c70b1b87a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.210774 May 24 10:15:27.834445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 129/234] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:27 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:27.845089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:27.845989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:27.853231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:27.854539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:27.855439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:27.855937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:27.883245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:15:28.333429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:28.334153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:28.341000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.341520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.342307 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.343369 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.350746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-90ab21e7-1543-4913-a0dc-320c7ff9a8df tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "DELETE /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 204 len: 0 microversion: 2.1 time: 0.509786 May 24 10:15:28.351780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 106/235] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:15:27 2022] DELETE /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:15:28.366150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:28.367313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:28.392440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.393297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.394307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.394736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.523862 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:28.534619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-efeca36e-176d-4159-8490-03c9ca0e94d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1503 microversion: 2.1 time: 0.205105 May 24 10:15:28.535271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 130/236] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:28 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1503 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:28.554073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "24a99c40-983c-4254-9778-b26869237d1a"}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:28.555593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:28.565687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:28.570795 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.571524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.572598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.573248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.576021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-925ba759-a0cb-4a25-b9ca-88f6f64b0d5f tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1502 microversion: 2.1 time: 0.219046 May 24 10:15:28.587465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 107/237] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:28 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1502 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:28.606603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:28.611851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] HTTP exception thrown: Cannot 'attach_volume' instance b3484367-cf43-4128-b7f2-96124543234c while it is in vm_state rescued May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Request method failure captured: May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/os-volume_attachments HTTP/1.1 May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 74 May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_Tzo8x-h618vQIK2AZLsdhbocu8prJbiBdNbF4qbpYG1T25CEKu0iXYE0uhR0ccC8E3yZqmtj1CZrNPnmvDbAL3DpOJ2ErogKclQ3hggHrFsUzfMzEWXvOyC-9cvzh23HWiH6chQJDjdzC48vA8e-TI3CS-ooI8UxDpF3wTWTNLUA May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:15:28.613753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: baac9b5e923747e6986720e3ac849056 May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {"volumeAttachment": {"volumeId": "24a99c40-983c-4254-9778-b26869237d1a"}} May 24 10:15:28.615308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Cannot 'attach_volume' instance b3484367-cf43-4128-b7f2-96124543234c while it is in vm_state rescued May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'server_id': 'b3484367-cf43-4128-b7f2-96124543234c', 'body': {'volumeAttachment': {'volumeId': '24a99c40-983c-4254-9778-b26869237d1a'}}} May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance b3484367-cf43-4128-b7f2-96124543234c in vm_state rescued. Cannot attach_volume while the instance is in this state. May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:15:28.616311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance b3484367-cf43-4128-b7f2-96124543234c while it is in vm_state rescued May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:15:28.617733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance b3484367-cf43-4128-b7f2-96124543234c while it is in vm_state rescued May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Returning 409 to user: Cannot 'attach_volume' instance b3484367-cf43-4128-b7f2-96124543234c while it is in vm_state rescued {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e8b2dbac-69b8-41b4-8505-c02d023432cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.067091 May 24 10:15:28.620394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 131/238] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:15:28 2022] POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/os-volume_attachments => generated 152 bytes in 70 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) May 24 10:15:28.635308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:28.636468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:28.666107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.668396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.672742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.674339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.754844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:28.851945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:28.852729 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:28.862679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.863253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:28.864321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:28.864886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:29.023737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:29.033710 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-27dd7edf-dfb0-4a86-832c-c9869e6c782d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.187232 May 24 10:15:29.034291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 132/239] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:28 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:29.085016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ecce0fdf-5beb-4662-8a03-6d7d73ff563f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/action" status: 202 len: 0 microversion: 2.1 time: 0.458860 May 24 10:15:29.087572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 108/240] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:15:28 2022] POST /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c/action => generated 0 bytes in 461 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:15:29.098558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:29.099280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:29.105855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:29.106408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:29.107300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:29.107819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:29.312569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:29.316545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a610b1d2-31b0-4f91-9a11-269765899c84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1511 microversion: 2.1 time: 0.222438 May 24 10:15:29.317123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 133/241] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:29 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1511 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:29.596312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:29.597004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:29.603876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:29.604330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:29.614031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:29.614031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:29.773880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:29.797788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ee982058-bc23-4829-ad1d-97b88b05ec24 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1502 microversion: 2.1 time: 0.191655 May 24 10:15:29.797788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 109/242] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:29 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1502 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:30.052122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:30.054858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:30.060455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:30.061662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:30.062637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:30.063258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:30.298167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:30.320887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b8fdeb5b-7d66-4184-89d9-cce261ba71a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1722 microversion: 2.1 time: 0.261516 May 24 10:15:30.323401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 134/243] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:30 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1722 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:30.335407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:30.336471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:30.347326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:30.348094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:30.349146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:30.357239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:30.644349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:30.654471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-855e250f-0e3f-47b0-b0ad-760acc61b8b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1511 microversion: 2.1 time: 0.323952 May 24 10:15:30.667140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 110/244] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:30 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1511 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:30.802992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:30.803809 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:30.810731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:30.812778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:30.813611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:30.814056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:31.001675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:31.011281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e99bceb9-c5ec-4d05-b553-797f0257e9e4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1502 microversion: 2.1 time: 0.213813 May 24 10:15:31.026811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 135/245] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:30 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1502 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:31.341329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:31.342228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:31.349552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:31.349948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:31.350756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:31.351169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:31.520921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:31.532594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d3da83b9-b3f3-49a5-bcee-3a25f027dba3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1699 microversion: 2.1 time: 0.194869 May 24 10:15:31.533949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 111/246] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:31 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1699 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:31.551201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-481cd309-f5a0-477e-843b-20497f4dedf2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:31.558931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-481cd309-f5a0-477e-843b-20497f4dedf2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:31.559380 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-481cd309-f5a0-477e-843b-20497f4dedf2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:31.626983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-481cd309-f5a0-477e-843b-20497f4dedf2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Going to try to start instance {{(pid=118233) start /opt/stack/nova/nova/compute/api.py:2730}} May 24 10:15:31.674016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:31.674641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:31.681279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:31.681747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:31.682524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:31.682961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:31.831240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:31.840336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-46bbcec9-cdda-4884-952e-9c10c43572a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1511 microversion: 2.1 time: 0.170784 May 24 10:15:31.841103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 112/247] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:31 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1511 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:32.001063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-481cd309-f5a0-477e-843b-20497f4dedf2 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.453424 May 24 10:15:32.012141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:32.013760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:32.015535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 136/248] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:15:31 2022] POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action => generated 0 bytes in 469 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:15:32.067280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:32.068546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:32.071588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.072080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:32.073206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.085370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:32.101678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.101924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:32.102744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.111090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:32.267609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:32.276961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-62fc90d3-c311-42d3-862f-286fc8356bba tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1460 microversion: 2.1 time: 0.229118 May 24 10:15:32.279585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 137/249] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:32 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1460 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:32.293766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-f63583aa-ef3e-4294-b9a1-b53ee516ade2 req-5b93746a-08ba-4292-8aed-9229a60a721b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7bb62cee-05f2-450d-96e7-ad37469c13e5", "name": "network-vif-deleted", "tag": "b9988d12-0125-4f5b-9219-74dda5e4ee6c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:32.309000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-f63583aa-ef3e-4294-b9a1-b53ee516ade2 req-5b93746a-08ba-4292-8aed-9229a60a721b service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.309428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-f63583aa-ef3e-4294-b9a1-b53ee516ade2 req-5b93746a-08ba-4292-8aed-9229a60a721b service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:32.324849 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-f63583aa-ef3e-4294-b9a1-b53ee516ade2 req-5b93746a-08ba-4292-8aed-9229a60a721b service nova] Creating event network-vif-deleted:b9988d12-0125-4f5b-9219-74dda5e4ee6c for instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 on hv-cinder-84311 May 24 10:15:32.330431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-f63583aa-ef3e-4294-b9a1-b53ee516ade2 req-5b93746a-08ba-4292-8aed-9229a60a721b service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049160 May 24 10:15:32.330965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 138/250] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:15:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:32.394403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:32.403143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fb57aadc-bbe6-408b-a923-1452d6161bb3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1708 microversion: 2.1 time: 0.395780 May 24 10:15:32.403726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 113/251] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:32 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1708 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:32.859865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:32.860568 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:32.867496 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.868012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:32.869328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:32.869832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:33.023022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:33.032355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7c1e2ef6-d528-452e-a580-abf2461d6a91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1511 microversion: 2.1 time: 0.177157 May 24 10:15:33.033137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 139/252] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:32 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1511 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:33.317933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:33.333667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:33.352086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:33.361233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:33.369856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:33.369856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:33.421233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:33.429962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:33.453625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:33.454177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:33.457556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:33.460454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:33.496109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:33.506827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4ad98e9-4ce8-4abd-afe5-4ec49345393d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1298 microversion: 2.1 time: 0.210717 May 24 10:15:33.508454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 114/253] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:33 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1298 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:33.633756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:33.643163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-425d262a-8bce-4d38-956a-3a5a723d4a5c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1708 microversion: 2.1 time: 0.225823 May 24 10:15:33.643742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 140/254] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:33 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1708 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:34.052904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:34.054379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:34.075684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.076952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.078005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.078745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.247209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:34.257110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96dac7c5-75b1-45d8-a20b-10bc4b56e91b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1511 microversion: 2.1 time: 0.209348 May 24 10:15:34.258408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 115/255] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:34 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1511 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:34.526230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:34.527557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:34.538302 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.538949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.540328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.540907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.575452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.577470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.577470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lazy-loading 'fault' on Instance uuid 7bb62cee-05f2-450d-96e7-ad37469c13e5 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:15:34.661758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:34.662499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:34.674487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.675247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.676100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:34.676592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:34.686531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:34.718062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-90966c53-6e8a-41e3-a6dc-700d4362c29e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 200 len: 1303 microversion: 2.1 time: 0.179985 May 24 10:15:34.718062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 141/256] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:34 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 1303 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:34.903734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:34.927204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-20375091-a402-417b-9520-2a11084ce0d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1708 microversion: 2.1 time: 0.270511 May 24 10:15:34.929836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 116/257] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:34 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1708 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:35.275919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:35.276501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:35.278637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:35.283463 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:35.283910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:35.284814 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:35.285248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:35.352317 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=4 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:35.438453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:35.446783 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9ad144b8-5113-4aa6-82f3-42e3812cb4b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1517 microversion: 2.1 time: 0.174991 May 24 10:15:35.447354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 142/258] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:35 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1517 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:35.734215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:35.734998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 7bb62cee-05f2-450d-96e7-ad37469c13e5] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:35.751793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:35.751793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:35.751793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:35.751793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:35.762602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] HTTP exception thrown: Instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 could not be found. May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Request method failure captured: May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 HTTP/1.1 May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLATZ5ONt4H635THWd8ga8ylv8jN39KlQ37gB-0vAhkubnwdMEW7GzAdxOHBa6H2ofv4qU7dlhHYfhP4z3uPEqhsBlomnvM7QgwO-D5_p3viNfp_xCEBZLkDuDdw8eAKAoP9kWnUXT4UO-izNQ92C_1YvivS74Ke_JjpL2fNqR5XhqE May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: a805d83a8e7747528285d543eb274ab6 May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-VolumesActionsTest-291498703 May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:15:35.765855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a805d83a8e7747528285d543eb274ab6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a805d83a8e7747528285d543eb274ab6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a805d83a8e7747528285d543eb274ab6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a805d83a8e7747528285d543eb274ab6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-VolumesActionsTest-291498703 May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: a805d83a8e7747528285d543eb274ab6 May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-VolumesActionsTest-291498703 May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-VolumesActionsTest-291498703-project May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 6e8f1cb9760f41c6b7c1eb379daf1b64 May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-VolumesActionsTest-291498703-project May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 could not be found. May 24 10:15:35.767091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '7bb62cee-05f2-450d-96e7-ad37469c13e5'} May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 could not be found. May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:15:35.767784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 could not be found. May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 could not be found. May 24 10:15:35.768901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:15:35.770980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Returning 404 to user: Instance 7bb62cee-05f2-450d-96e7-ad37469c13e5 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:15:35.772308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c04b6f0d-5c09-4a18-b473-6a91e58d356e tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5" status: 404 len: 111 microversion: 2.1 time: 0.042109 May 24 10:15:35.772941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 117/259] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:35 2022] GET /compute/v2.1/servers/7bb62cee-05f2-450d-96e7-ad37469c13e5 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:15:35.915536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-616009059", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "networks": [{"uuid": "8c0e95bb-fac1-46f0-9283-fbcc016bef68"}]}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:35.947432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:35.949024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:35.955380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:35.956122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:35.958014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:35.959152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:36.012681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] validate_networks() for [('8c0e95bb-fac1-46f0-9283-fbcc016bef68', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:15:36.199382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:36.211767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-02b43c5b-3a28-43ed-9cad-e8760bab25ee tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1708 microversion: 2.1 time: 0.268206 May 24 10:15:36.213343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 118/260] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:35 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1708 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:36.466522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:15:36.473180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:15:36.474040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:15:36.474638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:15:36.475218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:15:36.496225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:36.496678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:36.497516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:36.498078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:36.517012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Getting quotas for project a805d83a8e7747528285d543eb274ab6. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:15:36.521027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Getting quotas for user 6e8f1cb9760f41c6b7c1eb379daf1b64 and project a805d83a8e7747528285d543eb274ab6. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:15:36.532449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:15:36.534765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:15:36.704827 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-641b0833-9927-4107-b4c6-4db389566656 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.793557 May 24 10:15:36.705501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 143/261] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:15:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 795 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:15:36.718076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-510e3d34-5cbb-4f9e-ac54-1edd22b3b930 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:36.720670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-510e3d34-5cbb-4f9e-ac54-1edd22b3b930 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:36.770797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-510e3d34-5cbb-4f9e-ac54-1edd22b3b930 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:36.779254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-510e3d34-5cbb-4f9e-ac54-1edd22b3b930 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1213 microversion: 2.1 time: 0.067229 May 24 10:15:36.780407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 119/262] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:36 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1213 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:37.231329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:37.232076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:37.253779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:37.254794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:37.256797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:37.260369 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:37.271664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-02258351-9303-4924-ad14-f1901da6ebe4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'create', calling method: .version_select at 0x7f873252cee0>, body: {"keypair": {"name": "tempest-keypair-1192980076", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMflJBeb4btOO2xCPUpeWRLwFtUvd1JoU6M3km7ls4Q24cN4GqnY9Mt0nKT57GrwzlK0PRcwYeAyM8ze6gfJgvUFGTlAEmKARVTpYPekqoZ1ATDnEMFPjzKrRSaFmx2QUA=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:37.327743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-02258351-9303-4924-ad14-f1901da6ebe4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:15:37.333231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-02258351-9303-4924-ad14-f1901da6ebe4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for user baac9b5e923747e6986720e3ac849056 and project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:15:37.458644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-02258351-9303-4924-ad14-f1901da6ebe4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.189520 May 24 10:15:37.461599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 120/263] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:15:37 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:15:37.607323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:37.616071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-44afaa14-3aea-442b-b983-81e71e530bef tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1712 microversion: 2.1 time: 0.389733 May 24 10:15:37.616660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 144/264] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:37 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1712 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:37.798632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:37.799460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:37.812931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:37.813446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:37.814299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:37.814945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:37.898891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:37.912162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d66da916-e928-4b9a-b44b-7e61590f6b95 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1217 microversion: 2.1 time: 0.117939 May 24 10:15:37.913181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 121/265] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:37 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1217 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:38.932060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:38.932737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:38.941564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:38.942020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:38.943164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:38.943649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:39.027665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:39.036106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2585be4b-fd1a-4109-a33c-1117c0dd7ca4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1265 microversion: 2.1 time: 0.107808 May 24 10:15:39.042493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 145/266] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:38 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1265 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:40.069771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:40.074246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:40.081369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:40.081976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:40.082896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:40.083481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:40.169751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:40.179976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-65d3873f-813d-4d74-8717-0ec1c279ba18 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1271 microversion: 2.1 time: 0.118727 May 24 10:15:40.189454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 122/267] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:40 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1271 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:40.947531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-42099812", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d063cd42-5325-4a1c-ac6c-ec74b4b895ae"}], "security_groups": [{"name": "tempest-securitygroup--1048173926"}], "key_name": "tempest-keypair-1192980076", "user_data": "}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:41.135756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] validate_networks() for [('d063cd42-5325-4a1c-ac6c-ec74b4b895ae', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:15:41.198161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:41.201014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:41.207415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:41.208089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:41.208839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:41.209376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:41.346438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:41.355047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d6bec526-be23-4de8-982a-090f0b108f06 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.160802 May 24 10:15:41.355642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 123/268] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:41 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:41.699684 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:15:41.700113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:15:41.700662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:15:41.701077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:15:41.702281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:15:41.711299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:41.711752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:41.712588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:41.713029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:41.729780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:15:41.734125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Getting quotas for user baac9b5e923747e6986720e3ac849056 and project af776ddc4c0f4e6390301a02c6bbfca0. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:15:41.745103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:15:41.747366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:15:41.863332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-3c38caf6-2571-4157-8a76-93d36a90998a req-adc88222-0342-40dc-a9e2-e45706ccd80d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "076ccd9c-b2b7-40ad-9651-4a427e367870", "tag": "94a97802-62a2-4103-99d3-c548814d1082"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:41.872547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-3c38caf6-2571-4157-8a76-93d36a90998a req-adc88222-0342-40dc-a9e2-e45706ccd80d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:41.873899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-3c38caf6-2571-4157-8a76-93d36a90998a req-adc88222-0342-40dc-a9e2-e45706ccd80d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:41.886082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-3c38caf6-2571-4157-8a76-93d36a90998a req-adc88222-0342-40dc-a9e2-e45706ccd80d service nova] Creating event network-changed:94a97802-62a2-4103-99d3-c548814d1082 for instance 076ccd9c-b2b7-40ad-9651-4a427e367870 on hv-cinder-84311 May 24 10:15:41.892234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-3c38caf6-2571-4157-8a76-93d36a90998a req-adc88222-0342-40dc-a9e2-e45706ccd80d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035215 May 24 10:15:41.893315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 124/269] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:15:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:41.985778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-af1d0982-e2b4-48a2-8966-b079a2ca5219 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.043115 May 24 10:15:41.986745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 146/270] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:15:40 2022] POST /compute/v2.1/servers => generated 409 bytes in 1045 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:15:41.996794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-55e08960-5320-4457-b702-78a64ee1c98b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:41.997550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-55e08960-5320-4457-b702-78a64ee1c98b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:42.053849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55e08960-5320-4457-b702-78a64ee1c98b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:42.061882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-55e08960-5320-4457-b702-78a64ee1c98b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1244 microversion: 2.1 time: 0.068352 May 24 10:15:42.062474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 125/271] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1244 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:42.371961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:42.372578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:42.378946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:42.379398 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:42.380157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:42.380593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:42.516066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:42.523454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-83999c93-cd7b-4a69-840d-f51fd1974bbe tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.155491 May 24 10:15:42.524017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 147/272] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:42 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:43.079612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:43.080228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:43.087464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:43.087912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:43.088675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:43.089123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:43.149884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:43.157452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-72bb7599-8682-4e15-a9aa-4647ffa0f153 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1248 microversion: 2.1 time: 0.082515 May 24 10:15:43.158016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 126/273] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1248 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:43.540419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:43.540963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:43.548400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:43.548849 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:43.549638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:43.550075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:43.677892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:43.685309 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-61466b07-1026-4750-a093-af2bc85aa947 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.148753 May 24 10:15:43.686496 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 148/274] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:43 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:44.176775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:44.178083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:44.186639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:44.187497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:44.188562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:44.189375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:44.266215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:44.274202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7c344411-95ba-4656-b4fb-e94be0c5de6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1248 microversion: 2.1 time: 0.102783 May 24 10:15:44.274714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 127/275] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1248 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:44.703685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:44.710029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:44.710690 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:44.711140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:44.711897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:44.712329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:44.715036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:44.848423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:44.855905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8c4b2bf9-de4f-4128-a6dd-87899e6b28ee tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.156921 May 24 10:15:44.856466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 149/276] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:44 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:45.291467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:45.292012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:45.299370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:45.299816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:45.300567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:45.301025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:45.363429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:45.370885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a8ce7eec-7608-42bc-aed0-aee89f15c379 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1304 microversion: 2.1 time: 0.083586 May 24 10:15:45.371450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 128/277] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1304 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:45.873708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:45.874326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:45.881388 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:45.881841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:45.884851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:45.884851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:46.080201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:46.089239 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a953ac0c-0567-4a2e-8a55-90714db7a1d7 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.219741 May 24 10:15:46.102816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 150/278] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:45 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:46.388457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:46.389892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:46.396911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:46.397430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:46.398270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:46.398700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:46.402767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:46.463957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:46.471258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d0ab7214-9750-4baf-8470-cf15b107e2da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1314 microversion: 2.1 time: 0.087131 May 24 10:15:46.471818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 129/279] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1314 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:47.105934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:47.106548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:47.114978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:47.115430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:47.116201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:47.116644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:47.488937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:47.489704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:47.497523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:47.498764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:47.500122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:47.503531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:47.503651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:47.512324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-89620949-e27f-45a3-a934-4dd8a565fd3a tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.409957 May 24 10:15:47.512974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 151/280] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:47 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:47.568047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:47.575512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dc20f8e9-ecb7-4844-a4e6-61a71547cb6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1314 microversion: 2.1 time: 0.090955 May 24 10:15:47.576115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 130/281] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1314 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:48.531422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:48.532169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:48.540275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:48.540726 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:48.541583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:48.542016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:48.597826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:48.600114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:48.606576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:48.614152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:48.614152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:48.614152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:48.676593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:48.684151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6da65ab4-01e1-4546-9c00-98f57559f81f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1302 microversion: 2.1 time: 0.094844 May 24 10:15:48.684681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 131/282] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1302 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:48.688797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:48.696403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-550b7b48-544f-49db-90f4-1ebaef27e23d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.169769 May 24 10:15:48.696958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 152/283] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:48 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:49.704134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:49.705171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:49.712487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:49.713189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:49.720117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:49.720567 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:49.721542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:49.721978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:49.727455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:49.727823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:49.728937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:49.729761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:49.888217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:49.911773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-843b40e7-aaa5-45c8-8736-4f08e3c3ee12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.212147 May 24 10:15:49.925720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 132/284] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:50.008825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:50.017437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3dfb451c-1f44-4b39-bc66-0cdc3dc25e7d tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.308475 May 24 10:15:50.018054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 153/285] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:49 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:50.942650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:50.943317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:50.950866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:50.951372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:50.952142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:50.952583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:51.034996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:51.035790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:51.042638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:51.043190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:51.044023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:51.044524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:51.091107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:51.091107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-534a77d3-b088-4f00-b2a7-ae138207b280 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.149052 May 24 10:15:51.091107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 133/286] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:51.181513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:51.190040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2c3b0e1e-a1d9-4b18-89e6-90b8b1b9b12b tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.159024 May 24 10:15:51.190599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 154/287] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:51 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:52.053888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-f7e3cab6-5ce0-48c1-9e3c-fc90f3f795a7 req-5fb23e66-f40b-4bed-a6b0-892bc51ab903 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "79059316-a25c-4dd7-954c-0f7a413203a1", "tag": "6c9affb3-0667-4743-a73b-88352bae25e5"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:52.062735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=8 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:52.065158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-f7e3cab6-5ce0-48c1-9e3c-fc90f3f795a7 req-5fb23e66-f40b-4bed-a6b0-892bc51ab903 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:52.065614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-f7e3cab6-5ce0-48c1-9e3c-fc90f3f795a7 req-5fb23e66-f40b-4bed-a6b0-892bc51ab903 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:52.078412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-f7e3cab6-5ce0-48c1-9e3c-fc90f3f795a7 req-5fb23e66-f40b-4bed-a6b0-892bc51ab903 service nova] Creating event network-changed:6c9affb3-0667-4743-a73b-88352bae25e5 for instance 79059316-a25c-4dd7-954c-0f7a413203a1 on hv-cinder-84311 May 24 10:15:52.084490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-f7e3cab6-5ce0-48c1-9e3c-fc90f3f795a7 req-5fb23e66-f40b-4bed-a6b0-892bc51ab903 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034664 May 24 10:15:52.085090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 134/288] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:15:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:52.108432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:52.108432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:52.108432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=4 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:52.108432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=12 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:52.113214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:52.113693 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:52.114473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:52.114904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:52.207814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:52.208654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:52.216229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:52.217227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:52.218188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:52.220788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:52.296008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:52.306180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-06c1c96f-e6de-4235-bedb-7ffdef9e2cba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.205878 May 24 10:15:52.307192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 155/289] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:52.365518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:52.373777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-15f1c1f1-95f6-413c-9dd6-8b2f4ec2edd3 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.169609 May 24 10:15:52.374810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 135/290] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:52 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:53.021502 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-d8a49bea-6897-4ba2-af87-e93f413fb925 req-9984877e-c1a7-49b1-9d90-1ffe9dcae79d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "076ccd9c-b2b7-40ad-9651-4a427e367870", "name": "network-vif-plugged", "status": "completed", "tag": "94a97802-62a2-4103-99d3-c548814d1082"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:53.030791 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-d8a49bea-6897-4ba2-af87-e93f413fb925 req-9984877e-c1a7-49b1-9d90-1ffe9dcae79d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:53.031230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-d8a49bea-6897-4ba2-af87-e93f413fb925 req-9984877e-c1a7-49b1-9d90-1ffe9dcae79d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:53.042047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-d8a49bea-6897-4ba2-af87-e93f413fb925 req-9984877e-c1a7-49b1-9d90-1ffe9dcae79d service nova] Creating event network-vif-plugged:94a97802-62a2-4103-99d3-c548814d1082 for instance 076ccd9c-b2b7-40ad-9651-4a427e367870 on hv-cinder-84311 May 24 10:15:53.048236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-d8a49bea-6897-4ba2-af87-e93f413fb925 req-9984877e-c1a7-49b1-9d90-1ffe9dcae79d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031090 May 24 10:15:53.048809 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 156/291] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:15:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:15:53.324131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:53.325447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:53.332783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:53.333238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:53.334030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:53.334465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:53.393160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:53.393793 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:53.400122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:53.400577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:53.401375 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:53.401812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:53.474025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:53.483037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3d135927-376e-4f7a-b2db-45875c45e584 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.162995 May 24 10:15:53.493426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 136/292] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:53.556050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:53.566286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dc165805-fdcf-4eb8-9024-0af61e0e0747 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.178167 May 24 10:15:53.567166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 157/293] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:53 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:53.901609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-127022607", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "networks": [{"uuid": "0ecf4be9-cd66-415d-a00c-6313ce09660a"}]}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:15:53.953319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] validate_networks() for [('0ecf4be9-cd66-415d-a00c-6313ce09660a', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:15:54.314077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:15:54.314490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:15:54.315021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:15:54.315459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:15:54.315914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:15:54.325216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:54.325876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:54.327014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:54.327513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:54.346866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Getting quotas for project 0f0441599dc5456da3e0f48be7dd2d20. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:15:54.350984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Getting quotas for user bb2d87f281834e4697a8c3559b6e83a2 and project 0f0441599dc5456da3e0f48be7dd2d20. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:15:54.363108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:15:54.365426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:15:54.384949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:54.499457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c9da4eac-f5bb-41b7-9b3a-2e71d16ca3aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.602211 May 24 10:15:54.500267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 137/294] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:15:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 603 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:15:54.501096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:54.501862 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:54.509258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:54.509741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:54.509965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7d2b672a-9099-4783-88c1-6289ae3c40ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:54.510519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:54.510638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7d2b672a-9099-4783-88c1-6289ae3c40ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:54.510960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:54.573705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d2b672a-9099-4783-88c1-6289ae3c40ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:54.579683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7d2b672a-9099-4783-88c1-6289ae3c40ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1218 microversion: 2.1 time: 0.073554 May 24 10:15:54.592650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 138/295] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1218 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:54.605521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:54.605521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:54.605521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:54.605521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:54.606383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:54.606823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:54.687392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:54.701511 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-59fffe15-adfc-4eec-b02b-b5fa719da3fa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.204804 May 24 10:15:54.711745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 158/296] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:54.768906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:54.776546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-76706fa7-7f3f-41c3-8edf-5d3d4b4921a1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.182669 May 24 10:15:54.777117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 139/297] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:54 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:55.596164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-227d53bb-c14f-47e1-93fa-058ea66c2dcc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:55.596794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-227d53bb-c14f-47e1-93fa-058ea66c2dcc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:55.659655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-227d53bb-c14f-47e1-93fa-058ea66c2dcc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:55.660648 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-227d53bb-c14f-47e1-93fa-058ea66c2dcc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1218 microversion: 2.1 time: 0.068524 May 24 10:15:55.661207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 159/298] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1218 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:55.719140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:55.719746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:55.726524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:55.726970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:55.727785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:55.728219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:55.793981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:55.794609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:55.801865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:55.802404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:55.803174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:55.803663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:55.864062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:55.871860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a82485e9-362c-4e86-b576-fbd4557dade1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.156978 May 24 10:15:55.876074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 140/299] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:55.978541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:55.986978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e05ae731-abd8-43b4-a5f0-4636d857cc32 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.196904 May 24 10:15:55.987550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 160/300] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:55 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:56.680175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:56.684644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:56.688433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:56.689205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:56.690262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:56.690691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:56.759026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:56.767333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a39aa5e-b9f8-461f-aa4e-08e83bdb62ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1222 microversion: 2.1 time: 0.091412 May 24 10:15:56.767905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 141/301] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:56 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1222 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:56.888477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:56.889102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:56.896383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:56.896845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:56.897714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:56.898158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:57.004740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:57.005440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:57.011347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:57.011801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:57.012598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:57.013033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:57.036339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:57.044479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3f787c51-9de6-4ebd-99f7-660b7b82cbf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.160024 May 24 10:15:57.045053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 161/302] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:57.155012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:57.162562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5e60c99d-87ba-44e1-a60e-5b54122eebe1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.161805 May 24 10:15:57.163142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 142/303] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:56 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:57.784709 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:57.785341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:57.791429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:57.801431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:57.801431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:57.801431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:57.857574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:57.865261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b747426a-a660-4419-b34f-c99567c25471 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1222 microversion: 2.1 time: 0.085212 May 24 10:15:57.865881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 162/304] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:57 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1222 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:58.063572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:58.064197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:58.072164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:58.072653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:58.073533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:58.074098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:58.180685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:58.181369 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:58.188484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:58.189049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:58.189930 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:58.190431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:58.216037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:58.224036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-54685cda-be9b-4695-a46e-cbe1017e9a43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.165548 May 24 10:15:58.225958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 143/305] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:58.344557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:58.351975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f6ff08b6-2a33-4ca8-8f3b-99b9817e33e0 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.175640 May 24 10:15:58.355698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 163/306] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:58 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:58.883164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:58.883873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:58.890360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:58.893523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:58.893523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:58.893523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:58.957175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:58.964579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-656e9e4c-d56b-44a8-9af3-8286fdeff409 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1214 microversion: 2.1 time: 0.085500 May 24 10:15:58.965151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 144/307] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:58 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1214 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:59.243017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:59.244232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:59.251524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:59.251753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:59.252518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:59.252951 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:59.370198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:59.370835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:59.377901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:59.378354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:59.379131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:59.379565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:59.386588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:59.403658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-87d32e9a-66d4-46ce-82a2-14703991b585 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.158594 May 24 10:15:59.403658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 164/308] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:15:59.529850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:15:59.537439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0885e167-6a43-4364-899c-aa318eb71629 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.171170 May 24 10:15:59.538000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 145/309] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:59 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:15:59.984205 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:15:59.984820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:15:59.990921 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:59.991372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:15:59.992138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:15:59.992579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:00.056477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:00.064762 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c67723ed-3457-4cb0-bfc4-3c36317ad20a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1270 microversion: 2.1 time: 0.084863 May 24 10:16:00.065364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 165/310] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:15:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1270 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:00.418811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:00.419420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:00.427040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:00.427493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:00.428255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:00.428691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:00.555811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:00.556431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:00.558279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:00.563387 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:00.564011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:00.564971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:00.565554 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:00.566564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ccea922d-3c24-4e06-8536-cb54eb5337b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.151640 May 24 10:16:00.567287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 146/311] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:00.635820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-2144f19f-c943-43c5-b485-d5bd72f6af72 req-cfa2d129-f2aa-42ce-8633-5178ea583f5f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "79059316-a25c-4dd7-954c-0f7a413203a1", "name": "network-vif-plugged", "status": "completed", "tag": "6c9affb3-0667-4743-a73b-88352bae25e5"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:00.647640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-2144f19f-c943-43c5-b485-d5bd72f6af72 req-cfa2d129-f2aa-42ce-8633-5178ea583f5f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:00.647640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-2144f19f-c943-43c5-b485-d5bd72f6af72 req-cfa2d129-f2aa-42ce-8633-5178ea583f5f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:00.657170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-2144f19f-c943-43c5-b485-d5bd72f6af72 req-cfa2d129-f2aa-42ce-8633-5178ea583f5f service nova] Creating event network-vif-plugged:6c9affb3-0667-4743-a73b-88352bae25e5 for instance 79059316-a25c-4dd7-954c-0f7a413203a1 on hv-cinder-84311 May 24 10:16:00.663050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-2144f19f-c943-43c5-b485-d5bd72f6af72 req-cfa2d129-f2aa-42ce-8633-5178ea583f5f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031174 May 24 10:16:00.663638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 147/312] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:16:00.698345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:00.706524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5dc51604-e338-4f6c-a7c9-c8887491f738 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.155074 May 24 10:16:00.707090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 166/313] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:00 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:01.082793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:01.083532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:01.089803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:01.090250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:01.091015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:01.091446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:01.157122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:01.160715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.164954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4fb7ee62-a409-48a5-9210-a0e2e76d5790 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1270 microversion: 2.1 time: 0.087083 May 24 10:16:01.165551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 148/314] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1270 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:01.583568 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:01.584107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:01.590571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:01.591024 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:01.591790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:01.592230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:01.608461 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.722225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:01.724129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:01.724740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:01.729931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3e1c628c-1e2e-4c06-bcde-8f3c21581609 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.150322 May 24 10:16:01.730504 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 167/315] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:01.731663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:01.732170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:01.732946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:01.733406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:01.860185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:01.868704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-81ae8b2f-ea8f-46c6-8df0-737f12b48f75 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1313 microversion: 2.1 time: 0.149100 May 24 10:16:01.869290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 149/316] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:01 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1313 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:02.182590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:02.183204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:02.189553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:02.190038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:02.190805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:02.191236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:02.255076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:02.262597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2e518265-50e1-4967-bd17-3f9db72b8107 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1270 microversion: 2.1 time: 0.083822 May 24 10:16:02.263216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 168/317] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1270 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:02.747225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:02.747836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:02.755052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:02.755501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:02.756262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:02.756697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:02.886073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:02.886683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:02.889808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:02.893076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:02.895303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:02.895303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:02.895303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:02.897639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-be3b1cd4-a5fe-439d-8fab-5f04166a1027 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.154217 May 24 10:16:02.898209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 150/318] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:03.027581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:03.035506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8ac295cd-8089-4ce3-afb6-736016e79547 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1497 microversion: 2.1 time: 0.153368 May 24 10:16:03.036099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 169/319] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:02 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:03.279899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:03.280566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:03.287418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:03.291871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:03.291871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:03.291871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:03.354701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:03.362773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-afdfebe3-4793-4bc4-9c53-9360b24fc95c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1278 microversion: 2.1 time: 0.086775 May 24 10:16:03.363436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 151/320] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:03 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1278 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:03.914990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:03.915616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:03.922424 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:03.927287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:03.927287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:03.927287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:04.056297 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:04.064460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3932bef7-bfea-4aa7-a927-4645ff42a22e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.152907 May 24 10:16:04.065125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 170/321] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:04.380604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:04.381115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:04.383497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.388392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:04.388836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:04.389621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:04.390053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:04.452746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:04.460261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-276814f8-7c80-4d79-93c1-a7860a2a636d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1278 microversion: 2.1 time: 0.084135 May 24 10:16:04.460826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 152/322] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:04 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1278 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:05.081671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:05.082282 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:05.088924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:05.089210 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:05.090017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:05.090455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:05.226154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:05.233733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-010fd7a6-4828-4550-a8f6-e220b1634705 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.156386 May 24 10:16:05.237888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 171/323] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:05.477439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:05.477977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:05.479787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=8 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.485094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:05.485575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:05.486920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:05.487350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:05.550306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:05.557824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1cec5c62-034d-46b8-a74e-96383a08fc25 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1278 microversion: 2.1 time: 0.084584 May 24 10:16:05.558396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 153/324] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:05 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1278 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:06.182537 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-84020394", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-91633904", "security_groups": [{"name": "tempest-secgroup-smoke-328986262"}], "networks": [{"uuid": "5440374a-d63e-42a4-ab59-7628ba412e74"}], "block_device_mapping_v2": [{"uuid": "c789acbd-5415-4fd8-a87c-6059904e6fe6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:06.192315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:06.251200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:06.252120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:06.258555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.259001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.259766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.260210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.310704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] validate_networks() for [('5440374a-d63e-42a4-ab59-7628ba412e74', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:16:06.393260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:06.400812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1758eab-dc90-45e3-a59c-ab1be2531211 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.153550 May 24 10:16:06.410387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 154/325] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:06.506546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:06.507114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:06.513496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.513948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.514718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.515378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.546822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:16:06.790081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:16:06.790509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:16:06.790989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:16:06.791404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:16:06.791839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:16:06.802351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.803562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.803861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.804314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.822480 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Getting quotas for project 5bf4c198860648cd874233ba27d1b971. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:16:06.823473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5d9eb9fc-4de8-4942-806d-1f43fe2361e9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "DELETE /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 204 len: 0 microversion: 2.1 time: 0.321588 May 24 10:16:06.824576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:06.827052 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Getting quotas for user 577c7840904d4f2180f7e5bd0f530a9a and project 5bf4c198860648cd874233ba27d1b971. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:16:06.827876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 155/326] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:16:06 2022] DELETE /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 24 10:16:06.840230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:06.840959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:06.847534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.847980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.848930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:06.849383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:06.894720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:16:06.895291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c789acbd-5415-4fd8-a87c-6059904e6fe6',volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:16:07.005726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:07.014825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ceb95285-1912-4b13-a062-5899416bc2dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.185655 May 24 10:16:07.015402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 156/327] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:07.020983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:07.021985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:07.028784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:07.029509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:07.030491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:07.031103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:07.191879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:07.199553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c962f8f0-19c7-4ab2-9d0a-da786dc910b4 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1503 microversion: 2.1 time: 0.183003 May 24 10:16:07.200119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 157/328] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:07 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1503 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:07.424416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:07.425364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:07.427905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4a003c9b-29e4-48eb-b783-76ba5fd7a1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.249195 May 24 10:16:07.433980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 172/329] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:16:06 2022] POST /compute/v2.1/servers => generated 405 bytes in 1251 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:16:07.434961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:07.435511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:07.436471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:07.436974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:07.439962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ab7f555b-41a3-4d5d-bad9-d10b200b13c8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:07.442774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ab7f555b-41a3-4d5d-bad9-d10b200b13c8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:07.502978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ab7f555b-41a3-4d5d-bad9-d10b200b13c8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:07.509951 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ab7f555b-41a3-4d5d-bad9-d10b200b13c8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1085 microversion: 2.1 time: 0.074229 May 24 10:16:07.510527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 173/330] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1085 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:07.628533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:07.639887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4479f93a-adff-42e0-a7f5-8cdf9aa00d2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.220662 May 24 10:16:07.640662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 158/331] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:07.883191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-4bb03ed0-a5c4-4b5f-b72d-bec703610e7a req-b79cf959-f3f5-4f39-b284-0c7d6c50b03b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c4942f39-67fa-4c51-b915-7354d2c78f00", "tag": "55b9120e-64fd-43d8-bfff-e89f1183b17a"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:07.892668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-4bb03ed0-a5c4-4b5f-b72d-bec703610e7a req-b79cf959-f3f5-4f39-b284-0c7d6c50b03b service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:07.893244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-4bb03ed0-a5c4-4b5f-b72d-bec703610e7a req-b79cf959-f3f5-4f39-b284-0c7d6c50b03b service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:07.904879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-4bb03ed0-a5c4-4b5f-b72d-bec703610e7a req-b79cf959-f3f5-4f39-b284-0c7d6c50b03b service nova] Creating event network-changed:55b9120e-64fd-43d8-bfff-e89f1183b17a for instance c4942f39-67fa-4c51-b915-7354d2c78f00 on hv-cinder-84311 May 24 10:16:07.910158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-4bb03ed0-a5c4-4b5f-b72d-bec703610e7a req-b79cf959-f3f5-4f39-b284-0c7d6c50b03b service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030864 May 24 10:16:07.910887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 174/332] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:08.033066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:08.033758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:08.042605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:08.043237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:08.044039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:08.044973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:08.176733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:08.184371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-98dc8d3b-8c66-45c9-b67e-bff52eda1f21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.155197 May 24 10:16:08.184952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 159/333] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:08.218178 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:08.218867 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:08.226328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:08.226328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:08.226822 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:08.227261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:08.362646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:08.370782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1ec8d70a-376e-45ef-9836-7c85c66eaae9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1503 microversion: 2.1 time: 0.157348 May 24 10:16:08.371356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 175/334] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:08 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1503 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:08.527921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6d3d8cf-6705-49d8-a73f-fe563da87a44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:08.528660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f6d3d8cf-6705-49d8-a73f-fe563da87a44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:08.586242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f6d3d8cf-6705-49d8-a73f-fe563da87a44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:08.594560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f6d3d8cf-6705-49d8-a73f-fe563da87a44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1085 microversion: 2.1 time: 0.070336 May 24 10:16:08.595868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 160/335] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1085 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:08.658858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:08.659488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:08.680475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:08.683146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:08.685120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:08.686994 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:08.833330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:08.840871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4fd2c0ef-6d7f-494b-956e-fc5700c52986 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1370 microversion: 2.1 time: 0.186954 May 24 10:16:08.841466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 176/336] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1370 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:09.202458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:09.203068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:09.211801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.212254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.213061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.213693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.353036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:09.361346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6d6956ee-50da-4360-b8e3-97220f1d0a87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.162782 May 24 10:16:09.364003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 161/337] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:09.389980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:09.390678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:09.398565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.399040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.400072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.400758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.615067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:09.615683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:09.622010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.622463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.623309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.623744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.695435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:09.705097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-59caec19-bdc0-42d4-8c0b-98d95113415d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1135 microversion: 2.1 time: 0.095372 May 24 10:16:09.705726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 162/338] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1135 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:09.858601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:09.859175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:09.865748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.866208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.867132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:09.867577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:09.962385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:09.971110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b4a33e9f-2ba1-4134-8144-33e10ad91173 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1503 microversion: 2.1 time: 0.584430 May 24 10:16:09.971696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 177/339] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:09 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1503 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:10.014964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:10.023459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-37fbdcd9-500d-435d-8920-9663ced13f85 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1564 microversion: 2.1 time: 0.168951 May 24 10:16:10.024045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 163/340] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1564 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:10.040617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:10.041231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:10.047919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.049147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.050124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.050564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.183545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-eaca4479-2560-4646-ac95-e2d63244f42b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.146623 May 24 10:16:10.191798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 178/341] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:16:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-interface => generated 284 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:10.380695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:10.381381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:10.390570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.391022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.392316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.393117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.715144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-ec0756ed-10e5-4e0c-8832-8abf52916d88 req-5a172ee8-69e8-4004-b70c-ceda7bb2efdb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "076ccd9c-b2b7-40ad-9651-4a427e367870", "name": "network-vif-deleted", "tag": "94a97802-62a2-4103-99d3-c548814d1082"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:10.725990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-ec0756ed-10e5-4e0c-8832-8abf52916d88 req-5a172ee8-69e8-4004-b70c-ceda7bb2efdb service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.726446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-ec0756ed-10e5-4e0c-8832-8abf52916d88 req-5a172ee8-69e8-4004-b70c-ceda7bb2efdb service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.742268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-ec0756ed-10e5-4e0c-8832-8abf52916d88 req-5a172ee8-69e8-4004-b70c-ceda7bb2efdb service nova] Creating event network-vif-deleted:94a97802-62a2-4103-99d3-c548814d1082 for instance 076ccd9c-b2b7-40ad-9651-4a427e367870 on hv-cinder-84311 May 24 10:16:10.748374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-ec0756ed-10e5-4e0c-8832-8abf52916d88 req-5a172ee8-69e8-4004-b70c-ceda7bb2efdb service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038779 May 24 10:16:10.749483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 179/342] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:16:10.755661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:10.756378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:10.765474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.765941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.766762 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.767246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.858358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:10.867136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-72c39ebb-3464-4cfb-aa41-b9c63620927a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1191 microversion: 2.1 time: 0.116219 May 24 10:16:10.867992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 180/343] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1191 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:10.906462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:10.912504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:10.918372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-13213a58-203f-4bb0-b779-fdb1474d673d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.542910 May 24 10:16:10.919242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 164/344] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:10 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:10.988891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:10.989614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:10.996425 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.996887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:10.997724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:10.998168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:11.079712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:11.088460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1c75767f-5e1f-4718-b062-2844bd2dd1c8 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1298 microversion: 2.1 time: 0.104108 May 24 10:16:11.089028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 181/345] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:10 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1298 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:11.804492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-02db7b82-9b8b-4b25-bcff-570ab0c99739 req-38816660-985a-406b-9a88-6167606fb68d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "79059316-a25c-4dd7-954c-0f7a413203a1", "tag": "6c9affb3-0667-4743-a73b-88352bae25e5"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:11.817380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-02db7b82-9b8b-4b25-bcff-570ab0c99739 req-38816660-985a-406b-9a88-6167606fb68d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:11.817768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-02db7b82-9b8b-4b25-bcff-570ab0c99739 req-38816660-985a-406b-9a88-6167606fb68d service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:11.831663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-02db7b82-9b8b-4b25-bcff-570ab0c99739 req-38816660-985a-406b-9a88-6167606fb68d service nova] Creating event network-changed:6c9affb3-0667-4743-a73b-88352bae25e5 for instance 79059316-a25c-4dd7-954c-0f7a413203a1 on hv-cinder-84311 May 24 10:16:11.837752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-02db7b82-9b8b-4b25-bcff-570ab0c99739 req-38816660-985a-406b-9a88-6167606fb68d service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037184 May 24 10:16:11.838770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 165/346] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:11.893705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:11.894381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:11.922132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:11.940982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:11.941750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:11.944584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:11.945552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:11.946044 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:11.967275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:11.973052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:11.980185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:11.980863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:12.117962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:12.126863 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-67f1bdcb-f513-4ac6-bde0-8b4f77422d31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1201 microversion: 2.1 time: 0.245738 May 24 10:16:12.139228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 182/347] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1201 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:12.146257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:12.147189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:12.154076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:12.154755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:12.155778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:12.156418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:12.203844 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:12.204905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:12.205500 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lazy-loading 'fault' on Instance uuid 076ccd9c-b2b7-40ad-9651-4a427e367870 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:16:12.251698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:12.259264 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3281b9c4-f1c6-4d48-b00d-4581c75d9d59 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.324968 May 24 10:16:12.259856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 166/348] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:11 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:12.278926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e8f1cb9760f41c6b7c1eb379daf1b64', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a805d83a8e7747528285d543eb274ab6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:12.287830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8a4c07ac-7fb2-4540-b261-e46c8d34dfa9 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 200 len: 1303 microversion: 2.1 time: 0.146534 May 24 10:16:12.288431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 183/349] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:12 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 1303 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:13.151604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:13.152846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:13.167481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:13.168111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:13.174964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:13.179230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:13.277599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:13.278994 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:13.287438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:13.288264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:13.289290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:13.290085 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:13.461224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:13.467978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:13.471962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1af7c5be-2981-49e1-a98f-73e61d7ef042 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1268 microversion: 2.1 time: 0.325428 May 24 10:16:13.476486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-499f681d-4314-4a9d-b902-86c46654d0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.203598 May 24 10:16:13.477497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 184/350] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:13.482225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:13.483003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] [instance: 076ccd9c-b2b7-40ad-9651-4a427e367870] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:13.490728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 167/351] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1268 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:13.492574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:13.493034 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:13.493839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:13.494285 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:13.510821 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] HTTP exception thrown: Instance 076ccd9c-b2b7-40ad-9651-4a427e367870 could not be found. May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Request method failure captured: May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 HTTP/1.1 May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLATZ5ONt4H635THWd8ga8ylv8jN39KlQ37gB-0vAhkubnwdMEW7GzAdxOHBa6H2ofv4qU7dlhHYfhP4z3uPEqhsBlomnvM7QgwO-D5_p3viNfp_xCEBZLkDuDdw8eAKAoP9kWnUXT4UO-izNQ92C_1YvivS74Ke_JjpL2fNqR5XhqE May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: a805d83a8e7747528285d543eb274ab6 May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesActionsTest-291498703 May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:16:13.512498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a805d83a8e7747528285d543eb274ab6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a805d83a8e7747528285d543eb274ab6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a805d83a8e7747528285d543eb274ab6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a805d83a8e7747528285d543eb274ab6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesActionsTest-291498703 May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: a805d83a8e7747528285d543eb274ab6 May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesActionsTest-291498703 May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesActionsTest-291498703-project May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 6e8f1cb9760f41c6b7c1eb379daf1b64 May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesActionsTest-291498703-project May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Instance 076ccd9c-b2b7-40ad-9651-4a427e367870 could not be found. May 24 10:16:13.514171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '076ccd9c-b2b7-40ad-9651-4a427e367870'} May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 076ccd9c-b2b7-40ad-9651-4a427e367870 could not be found. May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:16:13.515923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 076ccd9c-b2b7-40ad-9651-4a427e367870 could not be found. May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 076ccd9c-b2b7-40ad-9651-4a427e367870 could not be found. May 24 10:16:13.517742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:16:13.520098 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] Returning 404 to user: Instance 076ccd9c-b2b7-40ad-9651-4a427e367870 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:16:13.520098 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6c685f1e-3c48-43e5-a606-427103a6cfb1 tempest-VolumesActionsTest-291498703 tempest-VolumesActionsTest-291498703-project] 10.222.0.35 "GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870" status: 404 len: 111 microversion: 2.1 time: 0.035400 May 24 10:16:13.520098 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 185/352] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:13 2022] GET /compute/v2.1/servers/076ccd9c-b2b7-40ad-9651-4a427e367870 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:16:14.489283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:14.490530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:14.496769 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:14.497405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:14.500544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:14.501810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:14.506650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:14.506650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:14.510319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:14.510319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:14.513393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:14.516517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:14.694399 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:14.706572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-eb502bd7-6140-474b-8a25-3160e0ba9f4e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.214376 May 24 10:16:14.717148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 186/353] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:14 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:14.742342 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-9890092f-f503-4808-b9a1-402e5914fc06 req-2380a84e-f2df-476d-9081-5cb3722ac5af service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "426e8866-95a1-42e7-9e88-29f93c5e7d7b", "tag": "0ed9cac0-6315-452a-9289-41b7a7958b3c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:14.768919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-9890092f-f503-4808-b9a1-402e5914fc06 req-2380a84e-f2df-476d-9081-5cb3722ac5af service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:14.769774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-9890092f-f503-4808-b9a1-402e5914fc06 req-2380a84e-f2df-476d-9081-5cb3722ac5af service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:14.795901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-9890092f-f503-4808-b9a1-402e5914fc06 req-2380a84e-f2df-476d-9081-5cb3722ac5af service nova] Creating event network-changed:0ed9cac0-6315-452a-9289-41b7a7958b3c for instance 426e8866-95a1-42e7-9e88-29f93c5e7d7b on hv-cinder-84311 May 24 10:16:14.822125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-9890092f-f503-4808-b9a1-402e5914fc06 req-2380a84e-f2df-476d-9081-5cb3722ac5af service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.108278 May 24 10:16:14.823431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 187/354] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:14.853841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:14.862194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-490e2eaa-81a5-4a4d-bbf9-e7c9052c8728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.377607 May 24 10:16:14.863891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 168/355] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:15.725369 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:15.726059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:15.732162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:15.732616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:15.733399 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:15.733838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:15.749018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:15.870478 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:15.879552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-77ffd4ad-3f63-4a3f-bbd5-3df7aaf5dfe0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.158995 May 24 10:16:15.880710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:15.881335 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 188/356] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:15.882173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:15.889045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:15.893494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:15.894735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:15.895219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:16.085446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:16.092876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fc2c8225-7043-461e-b5ac-b8557ede4033 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.216856 May 24 10:16:16.093669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 169/357] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:16.897914 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:16.898521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:16.901159 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:16.906100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:16.906552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:16.907341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:16.907777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:17.111790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:17.112675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:17.119830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:17.121240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:17.122188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:17.122715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:17.163153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:17.170884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6a4d3c6a-7c92-4ec7-8769-64ed3d5968bf tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.276707 May 24 10:16:17.171717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 189/358] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:17.287065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:17.295065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4f070be2-843d-4270-a4ab-f80b993e8d4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.187775 May 24 10:16:17.295653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 170/359] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:18.188615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:18.189911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:18.197175 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=8 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:18.200093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:18.200827 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:18.201828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:18.202452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:18.313195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:18.313914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:18.321125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:18.321575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:18.322552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:18.323054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:18.588004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:18.598271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:18.599566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:18.603703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-befdb138-4232-4a42-81df-216eb58f5f45 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.294555 May 24 10:16:18.604962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 171/360] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:18.612032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-0749688e-a170-4f98-83ef-9a647f890b58 req-03209fee-08e0-46f5-a912-cd58ec99bb3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4942f39-67fa-4c51-b915-7354d2c78f00", "name": "network-vif-plugged", "status": "completed", "tag": "55b9120e-64fd-43d8-bfff-e89f1183b17a"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:18.626830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fe61e586-3e25-4ca9-b2cd-6d17c1543f7b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.442710 May 24 10:16:18.628037 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 190/361] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:18 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:18.631416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-0749688e-a170-4f98-83ef-9a647f890b58 req-03209fee-08e0-46f5-a912-cd58ec99bb3e service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:18.631848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-0749688e-a170-4f98-83ef-9a647f890b58 req-03209fee-08e0-46f5-a912-cd58ec99bb3e service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:18.653333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-0749688e-a170-4f98-83ef-9a647f890b58 req-03209fee-08e0-46f5-a912-cd58ec99bb3e service nova] Creating event network-vif-plugged:55b9120e-64fd-43d8-bfff-e89f1183b17a for instance c4942f39-67fa-4c51-b915-7354d2c78f00 on hv-cinder-84311 May 24 10:16:18.660692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-0749688e-a170-4f98-83ef-9a647f890b58 req-03209fee-08e0-46f5-a912-cd58ec99bb3e service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052498 May 24 10:16:18.662432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 172/362] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:19.620402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:19.621113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:19.630005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:19.634899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:19.634899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:19.634899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:19.647408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:19.648531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:19.668574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:19.673777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:19.674983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:19.678339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:19.857891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:19.864361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:19.866218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-24941da6-57ce-48aa-926c-be720d70ac0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.249650 May 24 10:16:19.867653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 191/363] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:19.876507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cffc3f5f-2e66-40b1-a93c-1a42231b05b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.235258 May 24 10:16:19.879775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 173/364] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:19 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:20.885692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:20.886469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:20.897442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:20.898050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:20.905360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:20.907765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:20.908221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:20.908997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:20.909822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:20.912164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:20.912611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:20.914539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:21.176327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:21.188529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f1535224-133b-47c1-bb33-e7007b7f0652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1318 microversion: 2.1 time: 0.295174 May 24 10:16:21.189544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 174/365] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:20 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1318 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:21.307878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:21.319130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8b5ad6e8-7bf1-4414-98b7-9edca3d033fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.438657 May 24 10:16:21.331887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 192/366] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:22.220686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:22.220686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:22.228679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:22.229485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:22.230488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:22.231144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:22.540582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:22.540582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:22.540582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:22.540582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:22.544140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:22.544140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:22.544784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:22.544784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e96125ec-f7a3-4497-8ded-de7e5c44d9a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1507 microversion: 2.1 time: 0.272381 May 24 10:16:22.544784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 175/367] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1507 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:22.687010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:22.697131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6cdd898a-181e-4280-b415-860808553dec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.364542 May 24 10:16:22.707718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 193/368] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:23.714977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:23.715602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:23.725255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:23.726466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:23.728718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:23.730360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:23.899320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:23.908704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-50973f4e-8057-401e-9411-1a29b4c01553 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.197366 May 24 10:16:23.909308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 176/369] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:24.926965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:24.927723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:24.936438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:24.936896 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:24.937747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:24.938232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:25.214728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:25.225655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-abae7a7d-747a-4c60-8d00-4fa9cbc2ff27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.303844 May 24 10:16:25.227069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 194/370] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:26.243079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:26.254384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:26.263330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:26.264078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:26.265179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:26.265830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:26.629550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:26.638362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5a1e4850-e1f3-4742-ab8b-fc1501ed8b11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.399099 May 24 10:16:26.642712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 177/371] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:27.659278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:27.659891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:27.674031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:27.674487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:27.675279 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:27.675893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:27.950490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:27.962027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7a1dc751-af21-4f13-9ee1-3aec0734aee8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.306865 May 24 10:16:27.963506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 195/372] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:28.015383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:28.022640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:28.041813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:28.042751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:28.043720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:28.044915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:28.370725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lazy-loading 'services' on Instance uuid 4ca33d7e-37bd-4d91-8a41-866155df1bc3 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:16:28.780356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-3d5ba782-20df-49dd-8a86-3354ffd7a3b9 req-55f71ac7-7069-44d5-bbc6-e8dfec470cfa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "426e8866-95a1-42e7-9e88-29f93c5e7d7b", "name": "network-vif-plugged", "status": "completed", "tag": "0ed9cac0-6315-452a-9289-41b7a7958b3c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:28.812428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-3d5ba782-20df-49dd-8a86-3354ffd7a3b9 req-55f71ac7-7069-44d5-bbc6-e8dfec470cfa service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:28.813705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-3d5ba782-20df-49dd-8a86-3354ffd7a3b9 req-55f71ac7-7069-44d5-bbc6-e8dfec470cfa service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:28.843823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-3d5ba782-20df-49dd-8a86-3354ffd7a3b9 req-55f71ac7-7069-44d5-bbc6-e8dfec470cfa service nova] Creating event network-vif-plugged:0ed9cac0-6315-452a-9289-41b7a7958b3c for instance 426e8866-95a1-42e7-9e88-29f93c5e7d7b on hv-cinder-84311 May 24 10:16:28.855683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-3d5ba782-20df-49dd-8a86-3354ffd7a3b9 req-55f71ac7-7069-44d5-bbc6-e8dfec470cfa service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079467 May 24 10:16:28.856731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 196/373] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:28.880525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-78262c19-f2ac-4e15-942b-05da73ce4828 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-volume_attachments/a1037a38-e9c8-4a1f-8c9f-b661216734dc" status: 202 len: 0 microversion: 2.1 time: 0.877303 May 24 10:16:28.883638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 178/374] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:16:28 2022] DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-volume_attachments/a1037a38-e9c8-4a1f-8c9f-b661216734dc => generated 0 bytes in 882 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:16:28.983679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:28.984546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:29.012794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:29.012794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:29.012794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:29.012794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:29.390620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:29.405026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4531d564-cdfb-4f24-8c35-ec7608b910de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.426610 May 24 10:16:29.410668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 197/375] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:30.425544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:30.427122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:30.447894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=92,nova_api:UPDATE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:30.456674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:30.460048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:30.463152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:30.463599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:30.976558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:30.989109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b5dbcf27-c2c6-483d-a787-483f161c0350 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1256 microversion: 2.1 time: 0.568525 May 24 10:16:31.000947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 179/376] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1256 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:31.242745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f463faf8-3532-471b-8944-aeaefb0ce2ac tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:31.257019 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f463faf8-3532-471b-8944-aeaefb0ce2ac tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:31.258131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f463faf8-3532-471b-8944-aeaefb0ce2ac tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:31.358280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f463faf8-3532-471b-8944-aeaefb0ce2ac tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Going to try to stop instance {{(pid=118233) force_stop /opt/stack/nova/nova/compute/api.py:2707}} May 24 10:16:31.598132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f463faf8-3532-471b-8944-aeaefb0ce2ac tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.360365 May 24 10:16:31.626113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 198/377] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:16:31 2022] POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action => generated 0 bytes in 378 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:16:31.627346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:31.631523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:31.650565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:31.657943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:31.661290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:31.664530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:31.999286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:32.008050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:32.008991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:32.014367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-80a6f4c1-4eca-4484-9bf4-054f7fe388e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.395987 May 24 10:16:32.015358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 180/378] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:31 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:32.030836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:32.031386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:32.034233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:32.034678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:32.331761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:32.350303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c34d07e2-c39e-4494-91ad-15152ae5958d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1440 microversion: 2.1 time: 0.346526 May 24 10:16:32.351559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 199/379] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1440 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:32.370218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:32.381850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:32.466079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:32.468771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:32.485637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:32.486583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:32.897843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:32.908357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5586c8c4-f213-4ed0-a110-2eb441c7ee89 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1440 microversion: 2.1 time: 0.542610 May 24 10:16:32.914356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 181/380] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1440 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:33.035038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:33.035654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:33.044400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:33.049963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:33.050981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:33.051584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:33.398145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:33.407586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e7ed2568-4c77-4b3b-b720-eedfffaefb09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.375901 May 24 10:16:33.408736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 200/381] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:33 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:34.427183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:34.427801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:34.435050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:34.435668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:34.437516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:34.438152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:34.444430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:34.659905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:34.668771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7cf1a64a-24e1-49f1-ac2b-caeb18bd9f4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.247022 May 24 10:16:34.670005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 182/382] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:34 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:35.691565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:35.693807 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:35.717997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:35.719995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:35.724738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:35.728116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:35.979146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:35.996412 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-80941bc4-f6c1-4c6d-83e9-8d0e570faf9d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.309016 May 24 10:16:36.010438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 201/383] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:35 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:36.327077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-e54ba1fa-7ec6-4149-b48d-a3b83f740794 req-752de636-1501-4898-a069-743a98d3732f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "426e8866-95a1-42e7-9e88-29f93c5e7d7b", "tag": "0ed9cac0-6315-452a-9289-41b7a7958b3c"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:36.341784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-e54ba1fa-7ec6-4149-b48d-a3b83f740794 req-752de636-1501-4898-a069-743a98d3732f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:36.342229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-e54ba1fa-7ec6-4149-b48d-a3b83f740794 req-752de636-1501-4898-a069-743a98d3732f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:36.359066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-e54ba1fa-7ec6-4149-b48d-a3b83f740794 req-752de636-1501-4898-a069-743a98d3732f service nova] Creating event network-changed:0ed9cac0-6315-452a-9289-41b7a7958b3c for instance 426e8866-95a1-42e7-9e88-29f93c5e7d7b on hv-cinder-84311 May 24 10:16:36.365055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-e54ba1fa-7ec6-4149-b48d-a3b83f740794 req-752de636-1501-4898-a069-743a98d3732f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041272 May 24 10:16:36.370328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 183/384] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:37.022737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:37.024650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:37.034581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:37.037624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:37.038593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:37.047663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:37.341311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:37.350736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-93d99dc3-ff24-4c46-b061-74e65ed26017 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.337699 May 24 10:16:37.351780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 202/385] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:37 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:38.371415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:38.372510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:38.381995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:38.382520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:38.383415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:38.383903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:38.570676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:38.579160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c9510ffb-b423-4018-9c25-6302105555ea tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.212719 May 24 10:16:38.579750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 184/386] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:38 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:39.596481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:39.597105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:39.600800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=3 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:39.610831 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:39.611310 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:39.612117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:39.612564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:39.764807 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:39.772553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-df3de4a3-a0ec-43dc-a62b-c78a55fcc4ba tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.180102 May 24 10:16:39.773123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 203/387] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:39 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:40.794354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:40.795271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:40.811345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:40.812165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:40.815019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:40.822015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:41.039715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:41.050635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-999a16c3-fa4a-4206-9902-9aac8be033c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.262321 May 24 10:16:41.060610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 185/388] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:40 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:42.068926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:42.069596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:42.083381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:42.083847 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:42.084650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:42.085093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:42.315201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:42.317776 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:42.326300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c112b5a2-5363-4c36-b2f6-8b77bc29966f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.260966 May 24 10:16:42.327001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 204/389] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:42 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:43.345159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:43.345805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:43.352357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:43.352802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:43.353592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:43.354077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:43.522388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:43.553499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c7531133-f125-47d1-aa02-60f5ab5f9958 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.199911 May 24 10:16:43.554274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 186/390] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:43 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:44.562148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:44.563865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:44.572876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:44.573298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:44.574933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:44.575477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:44.755524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:44.763874 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d8e1aaa0-e37f-4611-9eee-a1fe2dcbbd4b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.206262 May 24 10:16:44.764531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 205/391] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:44 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:45.786292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:45.787715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:45.795607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:45.796379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:45.798746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:45.799406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:45.949615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:45.958438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd24cde7-a055-40c4-9165-4e687bb34f98 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.175917 May 24 10:16:45.958948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 187/392] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:45 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:46.976421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:46.977063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:46.988617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:46.989387 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:46.990351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:46.990969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:47.140595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:47.149720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-339067e2-d0ed-4bc3-b266-ffb02d14baa7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1676 microversion: 2.1 time: 0.177612 May 24 10:16:47.150656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 206/393] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:46 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1676 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:48.176604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:48.177219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:48.200257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:48.200844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:48.201666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:48.202182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:48.387748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:48.399411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6937781d-673e-4bf8-9489-f5e5aecdd61c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1653 microversion: 2.1 time: 0.227116 May 24 10:16:48.400257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 188/394] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:48 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1653 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:48.418057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1747417b-6a9d-48a2-b3d4-e3815783c3ae tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:48.425324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1747417b-6a9d-48a2-b3d4-e3815783c3ae tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:48.425706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1747417b-6a9d-48a2-b3d4-e3815783c3ae tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:48.456378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1747417b-6a9d-48a2-b3d4-e3815783c3ae tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Going to try to start instance {{(pid=118233) start /opt/stack/nova/nova/compute/api.py:2730}} May 24 10:16:48.684555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1747417b-6a9d-48a2-b3d4-e3815783c3ae tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action" status: 202 len: 0 microversion: 2.1 time: 0.270088 May 24 10:16:48.685466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 207/395] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:16:48 2022] POST /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/action => generated 0 bytes in 272 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:16:48.699874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:48.704603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:48.713187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:48.713660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:48.715049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:48.716402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:48.878258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:48.888662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-46ceed7e-6d1d-46ed-bdd0-7f78af23f978 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1662 microversion: 2.1 time: 0.196624 May 24 10:16:48.899225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 189/396] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:48 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1662 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:49.912296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:49.912779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:49.925008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:49.926131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:49.927462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:49.928228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:50.070958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:50.079142 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d8e8c209-d241-4526-aa1a-4764a10faed0 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1662 microversion: 2.1 time: 0.176280 May 24 10:16:50.079716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 208/397] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:49 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1662 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:50.460243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "816a261e-61a2-4f02-931e-d7d498967c0d", "device": "/dev/sdb"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:50.462077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:50.470185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:50.471166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:50.472167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:50.472661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:51.096806 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:51.097488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:51.103893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:51.104349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:51.105161 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:51.105627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:51.257439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:51.264964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d7e9889d-6e9f-4ffe-b9de-dc0b8444e099 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1666 microversion: 2.1 time: 0.172400 May 24 10:16:51.265565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 209/398] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:51 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1666 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:51.270113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-943fe695-de3f-4fbc-ac72-d1f8695b68b3 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Action: 'create', calling method: .version_select at 0x7f8732649e50>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-439592202", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKQOArMaXub5eU0FvfIYEopef4uoe7vFnmsNulLv8L7ZqOZo+i6g0Kk9UYB9CLeJQ6knxO2/NJocnL6f0DjR1QnqeI/t9hBcMH88nMu7ASqtQVfpUWRxHsRjQaxT+9Mfgg=="}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:51.284982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-943fe695-de3f-4fbc-ac72-d1f8695b68b3 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Getting quotas for project 2606d8cef9c442c896c12b401c92326f. Resources: {'key_pairs'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:16:51.288811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-943fe695-de3f-4fbc-ac72-d1f8695b68b3 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Getting quotas for user 77f178a1600c4704a9f7dba7e3b8f36a and project 2606d8cef9c442c896c12b401c92326f. Resources: {'key_pairs'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:16:51.377935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-943fe695-de3f-4fbc-ac72-d1f8695b68b3 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.111263 May 24 10:16:51.378674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 210/399] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:16:51 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:51.410942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-48c1274c-1241-4779-b23b-5acbd580638d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "POST /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.954552 May 24 10:16:51.411506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 190/400] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:16:50 2022] POST /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments => generated 194 bytes in 956 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:51.489072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1730667110", "imageRef": "157f6f63-62fd-4698-bb55-979585cc7512", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-439592202", "networks": [{"uuid": "3bc4cf08-8132-4e7d-b3cf-0d2b56babf28"}]}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:51.543763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] validate_networks() for [('3bc4cf08-8132-4e7d-b3cf-0d2b56babf28', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:16:51.934313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:16:51.934886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:16:51.935383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:16:51.939973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:16:51.939973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:16:51.945764 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:51.946300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:51.947096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:51.947531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:51.964541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Getting quotas for project 2606d8cef9c442c896c12b401c92326f. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:16:51.968623 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Getting quotas for user 77f178a1600c4704a9f7dba7e3b8f36a and project 2606d8cef9c442c896c12b401c92326f. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:16:51.980784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:16:51.983084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='157f6f63-62fd-4698-bb55-979585cc7512',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:16:52.091760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6de03017-81f1-47aa-9427-b82156df6cc8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.607200 May 24 10:16:52.092396 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 211/401] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:16:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 608 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:16:52.104099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-410654f5-f240-4b61-b1a4-39ded881633a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:52.104917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-410654f5-f240-4b61-b1a4-39ded881633a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:52.165156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-410654f5-f240-4b61-b1a4-39ded881633a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:52.175933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-410654f5-f240-4b61-b1a4-39ded881633a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1258 microversion: 2.1 time: 0.076604 May 24 10:16:52.176553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 191/402] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:52 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1258 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:16:53.193502 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3291b50a-fa47-40bc-927e-27ee430dbba8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:53.194121 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3291b50a-fa47-40bc-927e-27ee430dbba8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:53.248083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3291b50a-fa47-40bc-927e-27ee430dbba8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:53.256186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3291b50a-fa47-40bc-927e-27ee430dbba8 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1258 microversion: 2.1 time: 0.067134 May 24 10:16:53.256760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 212/403] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:53 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1258 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:54.273026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:54.273602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:54.280421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:54.280903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:54.281776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:54.282231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:54.352707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:54.360325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-24c15ed3-c41b-4736-bc52-726f5b64f25e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1310 microversion: 2.1 time: 0.091172 May 24 10:16:54.360939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 192/404] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:54 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1310 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:55.378688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:55.379514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:55.386785 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:55.387234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:55.388171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:55.388608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:55.532651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:55.540062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4ab410a1-4115-4198-ae6e-889ae86b8bd7 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.166338 May 24 10:16:55.540633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 213/405] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:55 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:56.401331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-acd692a6-ab83-4002-b0fe-9f335d530c05 req-85566ac5-34c2-41a8-b108-9c339a140f37 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6242a514-2829-4fa8-8572-96bca164b6a1", "tag": "a72bde82-4d88-4a59-8195-0159fccc9ac0"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:56.413776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-acd692a6-ab83-4002-b0fe-9f335d530c05 req-85566ac5-34c2-41a8-b108-9c339a140f37 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:56.414230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-acd692a6-ab83-4002-b0fe-9f335d530c05 req-85566ac5-34c2-41a8-b108-9c339a140f37 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:56.432098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-acd692a6-ab83-4002-b0fe-9f335d530c05 req-85566ac5-34c2-41a8-b108-9c339a140f37 service nova] Creating event network-changed:a72bde82-4d88-4a59-8195-0159fccc9ac0 for instance 6242a514-2829-4fa8-8572-96bca164b6a1 on hv-cinder-84311 May 24 10:16:56.437203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-acd692a6-ab83-4002-b0fe-9f335d530c05 req-85566ac5-34c2-41a8-b108-9c339a140f37 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040345 May 24 10:16:56.438876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 193/406] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:16:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:16:56.569943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:56.570570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:56.577118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:56.577610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:56.579250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:56.579899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:56.733687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:56.742644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-883a764f-c5ce-4bd6-b098-58442e8acfa6 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.176404 May 24 10:16:56.743372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 214/407] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:56 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:57.760983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:57.761857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:57.769776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:57.770233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:57.771033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:57.771465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:57.918474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:57.927097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e31696f7-1f74-4466-820f-e94b0364c949 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.170768 May 24 10:16:57.927682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 194/408] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:57 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:58.944427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:16:58.945851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:58.955959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:58.956421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:58.957204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:58.957672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:59.120871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:16:59.133171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-99d00304-cd9b-4a8c-b02d-4d8e84a042c9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.192084 May 24 10:16:59.133842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 215/409] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:16:58 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:16:59.333466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3bc1cf9a-17de-4332-b457-fa4e80a5fe7f"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:16:59.335361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:16:59.342538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:59.343065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:16:59.344048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:16:59.344463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:00.150666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:00.151454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:00.159528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:00.160280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:00.161258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:00.161900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:00.314778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:00.323749 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f6bd5fca-0488-47f1-b3db-5a1d23bcb339 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.177575 May 24 10:17:00.324360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 216/410] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:00 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:17:01.341043 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:01.341753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:01.349172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:01.349735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:01.350561 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:01.351027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:01.498230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:01.506063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-11524ba2-3426-45da-9d92-4c221b05a129 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.168960 May 24 10:17:01.506622 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 217/411] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:01 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:17:02.169773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:02.523961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:02.525316 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:02.531592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:02.534190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:02.534698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:02.535666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:02.536255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:02.688139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:02.696761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7e1bb292-f617-4bea-87be-87cc814f5b63 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.176939 May 24 10:17:02.697298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 218/412] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:02 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:17:03.715881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:03.717663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:03.720357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=12 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:03.726247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:03.729417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:03.729417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:03.729417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:03.876712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:03.884744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0087388e-95d1-48de-b6c2-bf9b68153830 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.173018 May 24 10:17:03.893341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 219/413] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:03 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:04.903148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:04.904020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:04.912000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:04.912536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:04.913493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:04.914060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:05.052300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:05.060107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8814a820-cb96-4452-829b-8ff901ab8c2b tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.160988 May 24 10:17:05.060892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 220/414] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:04 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:17:06.078603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:06.079985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:06.086502 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:06.086954 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:06.087718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:06.088156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:06.221627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:06.228915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ae70e271-617f-4359-bbf0-08cce112585f tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.154121 May 24 10:17:06.229519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 221/415] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:06 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:17:07.092465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:07.248308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:07.248960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:07.257547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:07.257801 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:07.258770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:07.259211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:07.403916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:07.412776 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e5dc5faa-d9f3-4cc7-b849-8271dbbe01fe tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.170127 May 24 10:17:07.413574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 222/416] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:07 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:17:07.654736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-190ff4b2-591a-4da6-a09c-d9170e2af4b1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 8.325094 May 24 10:17:07.655333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 195/417] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:16:59 2022] POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 194 bytes in 8327 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:17:08.430252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:08.430883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:08.437834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:08.438813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:08.439688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:08.440128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:08.444174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:08.591149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:08.600119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-db49910e-0800-4cb8-973d-5074b060da90 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.174131 May 24 10:17:08.600819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 223/418] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:08 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:09.618175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:09.620818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:09.621231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=35 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:09.626292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:09.626721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:09.627474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:09.627905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:09.770511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:09.778006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b1b82154-ede2-45f0-a016-037872165634 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.164077 May 24 10:17:09.778579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 196/419] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:09 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:10.797154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:10.798619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:10.800980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=4 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:10.806129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:10.809950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:10.809950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:10.809950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:10.945844 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:10.954810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c599bab1-4d58-49be-8dce-83296d4d3344 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.161478 May 24 10:17:10.955462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 224/420] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:10 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:11.971927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:11.972532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:11.979025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:11.979472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:11.980223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:11.980650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:12.106673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:12.114282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7881dfe5-205c-4b4c-b111-cfb62be97b99 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.146914 May 24 10:17:12.114847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 197/421] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:11 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:12.391240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-82a70fd0-668d-4061-af7d-52f960a6e51f req-f09c80ff-f5e3-498c-a900-45587e0eca82 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6242a514-2829-4fa8-8572-96bca164b6a1", "name": "network-vif-plugged", "status": "completed", "tag": "a72bde82-4d88-4a59-8195-0159fccc9ac0"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:17:12.400596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-82a70fd0-668d-4061-af7d-52f960a6e51f req-f09c80ff-f5e3-498c-a900-45587e0eca82 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:12.401096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-82a70fd0-668d-4061-af7d-52f960a6e51f req-f09c80ff-f5e3-498c-a900-45587e0eca82 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:12.413661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-82a70fd0-668d-4061-af7d-52f960a6e51f req-f09c80ff-f5e3-498c-a900-45587e0eca82 service nova] Creating event network-vif-plugged:a72bde82-4d88-4a59-8195-0159fccc9ac0 for instance 6242a514-2829-4fa8-8572-96bca164b6a1 on hv-cinder-84311 May 24 10:17:12.418795 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-82a70fd0-668d-4061-af7d-52f960a6e51f req-f09c80ff-f5e3-498c-a900-45587e0eca82 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031336 May 24 10:17:12.419602 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 225/422] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:17:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:17:13.131263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:13.131858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:13.138346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:13.138801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:13.139556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:13.139984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:13.265666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:13.274181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-572b10a3-4ed0-4e98-a2b1-84444fe9474e tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.146701 May 24 10:17:13.274777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 198/423] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:13 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:14.292499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:14.293191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:14.300047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:14.300496 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:14.301260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:14.301727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:14.435293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:14.454680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6e53d8d9-6a45-4c08-9555-f1a11c4e3984 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1358 microversion: 2.1 time: 0.166403 May 24 10:17:14.456941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 226/424] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:14 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:15.471741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:15.472373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:15.478929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:15.479374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:15.480134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:15.480566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:15.617472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:15.624809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-26747d58-5a79-4e87-a9a2-037929f86ed5 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1548 microversion: 2.1 time: 0.157508 May 24 10:17:15.625392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 199/425] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:15 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1548 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:15.641201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:15.642480 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:15.648781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:15.649239 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:15.650043 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:15.650490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:15.778676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:15.786773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-67a7841e-93d2-482f-a49b-e924eff1cba9 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1548 microversion: 2.1 time: 0.149359 May 24 10:17:15.787338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 227/426] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:15 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1548 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:15.802647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:15.806070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:15.806821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Searching by: {'deleted': False, 'project_id': '2606d8cef9c442c896c12b401c92326f'} {{(pid=118232) get_all /opt/stack/nova/nova/compute/api.py:2962}} May 24 10:17:15.813341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:15.813874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:15.814653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:15.815080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:15.905374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.multi_cell_list [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=118232) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} May 24 10:17:15.997567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-84cc6aa8-ad8c-444d-9222-cbaf38f54b6c tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.198659 May 24 10:17:15.998070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 200/427] 10.222.0.35 () {58 vars in 1124 bytes} [Tue May 24 10:17:15 2022] GET /compute/v2.1/servers => generated 343 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:17:16.008861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:16.012876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:16.015838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:16.016298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:16.017900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:16.018351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:16.146220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:16.153706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c607d23d-de34-4de2-913f-715caeeecfd4 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1548 microversion: 2.1 time: 0.148637 May 24 10:17:16.154349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 228/428] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:16 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1548 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:32.687181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:32.689088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:32.692300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=10 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:32.693366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:32.694389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=36,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:32.702806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:32.703278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:32.705082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:32.705555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:32.746633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1953dac3-1aed-4902-b9a7-922abc5c99cd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.063030 May 24 10:17:32.747217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 201/429] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:17:32 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3/os-volume_attachments => generated 25 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 24 10:17:32.806801 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:32.807389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:32.810589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=9 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:32.811340 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=31 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:32.817074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:32.823486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:32.823486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:32.823486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:32.851023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Going to try to terminate instance {{(pid=118233) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:17:33.129077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a69d9b95-d40a-482c-876b-e62beaa94cab tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 204 len: 0 microversion: 2.1 time: 0.326149 May 24 10:17:33.129702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 229/430] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:17:32 2022] DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 24 10:17:33.137712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:33.140071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:33.146406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:33.146850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:33.147614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:33.148042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:33.277238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:33.285126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9a8fe983-edc5-49d5-b736-888260d20aaa tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.151165 May 24 10:17:33.285720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 202/431] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:33 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:34.304092 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:34.304782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:34.311253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:34.311699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:34.312455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:34.312884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:34.447038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:34.454494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9ed87108-1814-46af-b91a-8583ea7c5f47 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.154694 May 24 10:17:34.455219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 230/432] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:34 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:35.472970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:35.472970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:35.478995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:35.479438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:35.480205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:35.480635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:35.607822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:35.615354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c8b01f38-df68-4bc5-84c9-6027b9b79e8f tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.147753 May 24 10:17:35.615916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 203/433] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:35 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:36.632107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:36.632724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:36.639144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:36.639590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:36.640355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:36.640788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:36.769230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:36.776778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9fd05a59-001e-4412-8899-d7312c48a8e8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.148872 May 24 10:17:36.777359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 231/434] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:36 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:37.795371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:37.795883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:37.803606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:37.804050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:37.804802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:37.805231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:37.938272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:37.945975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-848b8752-7302-444d-8ada-0d17140cb952 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.154613 May 24 10:17:37.946546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 204/435] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:37 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:38.963297 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:38.963902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:38.970632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:38.971079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:38.971842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:38.972275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:39.099754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:39.107296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-478e8e70-810b-4a66-8fc2-0f5fce1c1db1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.147785 May 24 10:17:39.107858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 232/436] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:38 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:40.123763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:40.124368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:40.131085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:40.131530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:40.132279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:40.132707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:40.260406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:40.267910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8e30415a-0071-4e3a-95f1-faee2a036e66 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.147865 May 24 10:17:40.268906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 205/437] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:40 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:41.286342 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:41.286906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:41.293193 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:41.293667 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:41.294420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:41.294850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:41.425588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:41.432920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-78f66c23-8b09-4639-b400-3e80d1b2e36d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.151290 May 24 10:17:41.433833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 233/438] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:41 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:42.449729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:42.450417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:42.457896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:42.458411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:42.459269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:42.459706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:42.589437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:42.596770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-400a83ee-0d1a-48dd-a774-c57734c2408e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.151003 May 24 10:17:42.597362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 206/439] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:42 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:43.614689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:43.615324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:43.621959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:43.622411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:43.623175 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:43.623611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:43.748736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:43.756244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1a0a9bd5-c4f6-406a-a781-dbaa566abafd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.146027 May 24 10:17:43.756805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 234/440] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:43 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:44.773939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:44.774484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:44.781010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:44.781566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:44.782367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:44.782799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:44.908912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:44.916478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7b9dbf55-b115-48c7-9dc5-2c4e92c71749 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.147122 May 24 10:17:44.917104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 207/441] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:44 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:45.934242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:45.934919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:45.941945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:45.942557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:45.943315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:45.943747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:46.076753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:46.085028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-86cced18-3476-4116-98ce-2652f37f72b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.154498 May 24 10:17:46.085648 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 235/442] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:45 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:47.103007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:47.103612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:47.110113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:47.110563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:47.111322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:47.111756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:47.242473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:47.249808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-da64599d-686e-4b72-a472-adf89124f9a7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.150722 May 24 10:17:47.254155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 208/443] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:47 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:48.267726 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:48.268447 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:48.275691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:48.276181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:48.277018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:48.277442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:48.406470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:48.414060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fc584879-eb82-4952-af47-99e8a4f58ecb tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.150390 May 24 10:17:48.414628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 236/444] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:48 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:49.431241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:49.431847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:49.438373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:49.438821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:49.439590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:49.440024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:49.571569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:49.579026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-23772899-67a2-4206-a280-a2125d87bf3d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.151546 May 24 10:17:49.579585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 209/445] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:49 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:50.596627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:50.597230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:50.604409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:50.604853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:50.605637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:50.606068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:50.736521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:50.744105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1673e920-b402-41fd-b573-f5b4005d0804 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.151304 May 24 10:17:50.744666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 237/446] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:50 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:51.761244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:51.761800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:51.768475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:51.768925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:51.770230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:51.770684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:51.896685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:51.904094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-038d8d3c-f280-44e0-9334-8d5af8696d17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.146821 May 24 10:17:51.904661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 210/447] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:51 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:52.921010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:52.921649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:52.928903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:52.929385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:52.930152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:52.930589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:53.060569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:53.068758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c973f943-d4ad-4bba-a1c6-4b50ebb13880 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.151580 May 24 10:17:53.069528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 238/448] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:52 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:54.087106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:54.087747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:54.094003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:54.094453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:54.095251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:54.095681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:54.230164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:54.237622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-abea3d7c-b8b3-4f39-8cc5-8a275c7bd97a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.154595 May 24 10:17:54.238199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 211/449] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:54 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:55.254646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:55.255286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:55.261927 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:55.262382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:55.263141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:55.263578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:55.417042 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:55.424723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bd41418a-a1dd-466e-8927-a7c1845e4619 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.173913 May 24 10:17:55.425308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 239/450] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:55 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:56.441363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:56.442205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:56.449360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:56.449804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:56.450563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:56.450992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:56.582648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:56.590280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-585787dd-7cfb-48ac-89c9-b2f280117f37 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.152517 May 24 10:17:56.590853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 212/451] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:56 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:57.607913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:57.608539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:57.615655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:57.616105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:57.616897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:57.617389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:57.751275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:57.759673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-252b0cbc-70c7-4a7b-90da-7cccc33b1b6c tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.155478 May 24 10:17:57.760313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 240/452] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:57 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:58.776682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:17:58.777609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:58.784558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:58.785085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:58.786220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:58.786747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:59.075900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:17:59.084098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1607f64a-cd0a-4454-9bac-6452a2d86d1d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1672 microversion: 2.1 time: 0.311332 May 24 10:17:59.084665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 213/453] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:17:58 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1672 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:17:59.900203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:17:59.902374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:17:59.920698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-a9f4dfad-e6ba-4c1d-b727-058c8bbedc7e req-51b76141-1ff6-43e3-83be-c416d4225ad8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca33d7e-37bd-4d91-8a41-866155df1bc3", "name": "network-vif-deleted", "tag": "697741fe-a360-4dd6-b2c9-a761d441a98d"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:17:59.927497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:59.927946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:59.928645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-a9f4dfad-e6ba-4c1d-b727-058c8bbedc7e req-51b76141-1ff6-43e3-83be-c416d4225ad8 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:59.928775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:17:59.929108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-a9f4dfad-e6ba-4c1d-b727-058c8bbedc7e req-51b76141-1ff6-43e3-83be-c416d4225ad8 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:59.929237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:17:59.941246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-a9f4dfad-e6ba-4c1d-b727-058c8bbedc7e req-51b76141-1ff6-43e3-83be-c416d4225ad8 service nova] Creating event network-vif-deleted:697741fe-a360-4dd6-b2c9-a761d441a98d for instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 on hv-cinder-84311 May 24 10:17:59.948217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-a9f4dfad-e6ba-4c1d-b727-058c8bbedc7e req-51b76141-1ff6-43e3-83be-c416d4225ad8 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034785 May 24 10:17:59.949372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 214/454] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:17:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:17:59.956706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lazy-loading 'flavor' on Instance uuid 79059316-a25c-4dd7-954c-0f7a413203a1 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:18:00.102659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:00.103276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:00.110055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:00.110672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:00.111284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:00.111724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:00.183851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:00.192127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96afaf7e-b857-4fe6-a1c0-3d94138e9627 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 200 len: 1322 microversion: 2.1 time: 0.094247 May 24 10:18:00.195679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 215/455] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:00 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 1322 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:00.234019 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:00.305747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b25c087-c311-4577-b33e-69d28b78ff14 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/action" status: 200 len: 32 microversion: 2.1 time: 0.409631 May 24 10:18:00.306501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 241/456] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:17:59 2022] POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/action => generated 32 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) May 24 10:18:00.315501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:00.316112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:00.322771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:00.323216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:00.323999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:00.324421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:00.452247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:00.462927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cfe787e3-c351-4908-b7e8-bfc7753d12cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.151218 May 24 10:18:00.463715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 216/457] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:01.211411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:01.212125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:01.220030 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:01.220486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:01.221533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:01.221977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:01.237848 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] HTTP exception thrown: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Request method failure captured: May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 HTTP/1.1 May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_TJiJC50rIjdcS63u7xuYhM4UlnZwxRv8mUKp0BzcDusYujaMOjfiQ7B-7czQB8mmEz41Y8c7oBuVaj_T84-448VkgY0Pf5gNrx-P1niD879m3w66ryHsb9xDN21s3gpWLUkY4irhyPqPHyYMccDT3Xqb0fUHM9qSZRWOI7k2kkeY May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:18:01.240522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-AttachVolumeTestJSON-1912151979 May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 7bffd469507b4dbc8d3146f051c8c8ad May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:18:01.242147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '4ca33d7e-37bd-4d91-8a41-866155df1bc3'} May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:18:01.243031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:18:01.244491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:18:01.245808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Returning 404 to user: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:18:01.245808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c5a82331-c075-46b1-84c4-40f592fe8c72 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 404 len: 111 microversion: 2.1 time: 0.036335 May 24 10:18:01.245808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 242/458] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:01 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:18:01.251585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0d24c27-0691-4166-beaa-c6624d79eacd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '.version_select at 0x7f87324d7b80>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:01.332834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0d24c27-0691-4166-beaa-c6624d79eacd tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-392535260" status: 202 len: 0 microversion: 2.1 time: 0.084426 May 24 10:18:01.337130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 217/459] 10.222.0.35 () {58 vars in 1217 bytes} [Tue May 24 10:18:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-392535260 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:18:01.480570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:01.481211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:01.488075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:01.488529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:01.489322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:01.489757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:01.675718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:01.685905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8602ed48-c1a7-4ee1-9ef7-821519c7f575 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.209417 May 24 10:18:01.689112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 243/460] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:02.153100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9cfa373a-8a9d-427f-83b2-06996dad64b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'create', calling method: .version_select at 0x7f8732407d30>, body: {"keypair": {"name": "tempest-keypair-1169943927", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKy7/zZSIlPwB2Mfcp+atpNCl5VmHgNSRzp6I/RRN87BTZ4g7qdPsxHneG9ga+V519e1BjwbwCGrPrQyBdHM46KeC//9UxL9WPK81lnJ8bEsQ/EOtOqIfW/mz0dv29RYlg=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:02.168348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-9cfa373a-8a9d-427f-83b2-06996dad64b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:18:02.172719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-9cfa373a-8a9d-427f-83b2-06996dad64b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for user 7bffd469507b4dbc8d3146f051c8c8ad and project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:18:02.285099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9cfa373a-8a9d-427f-83b2-06996dad64b6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.135843 May 24 10:18:02.286286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 218/461] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:18:02 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:02.706413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:02.707029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:02.714014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:02.715663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:02.715663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:02.715663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:02.852653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:02.860767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b5320092-4eea-4ffe-bab3-852f6056be30 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.158648 May 24 10:18:02.861654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 244/462] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:03.878247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:03.878859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:03.881223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=47,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:03.886267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:03.886796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:03.887562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:03.888002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:03.891966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=93 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:04.114898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:04.125046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-52cee933-6bfb-4dbb-9839-dd31f326eed2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.250824 May 24 10:18:04.125878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 219/463] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:04.780625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1610569307", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c3adfbde-72f1-4452-b19b-552144101170"}], "security_groups": [{"name": "tempest-securitygroup--1243571595"}], "key_name": "tempest-keypair-1169943927", "user_data": "}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:04.788150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:04.883675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.network.neutron [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] validate_networks() for [('c3adfbde-72f1-4452-b19b-552144101170', None, None, None, None, None)] {{(pid=118233) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:18:05.143127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:05.144079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:05.152787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:05.153391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:05.154298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:05.155301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:05.250559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Flavor limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:18:05.250559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Image limits 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:18:05.250559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Flavor pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:18:05.250846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Image pref 0:0:0 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:18:05.251148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.virt.hardware [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118233) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:18:05.261480 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:05.261968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:05.262771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:05.263207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:05.280194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:18:05.284144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.quota [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Getting quotas for user 7bffd469507b4dbc8d3146f051c8c8ad and project a02e2e12ebd14e5d8e960a65f41bfb00. Resources: {'cores', 'instances', 'ram'} {{(pid=118233) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:18:05.296021 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Going to run 1 instances... {{(pid=118233) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:18:05.298621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118233) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:18:05.323171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:05.331184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-548d530e-8bf4-47e0-bd52-0fcfd3dac0c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.192322 May 24 10:18:05.331747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 220/464] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:05.444798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c42990ee-23c5-4b4d-8666-5ae0a41e800b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.668330 May 24 10:18:05.445515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 245/465] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:18:04 2022] POST /compute/v2.1/servers => generated 402 bytes in 670 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:18:05.455450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1a5f1330-b618-4080-a626-fe94349f47d8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:05.461116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1a5f1330-b618-4080-a626-fe94349f47d8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:05.512078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1a5f1330-b618-4080-a626-fe94349f47d8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:05.519129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1a5f1330-b618-4080-a626-fe94349f47d8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1238 microversion: 2.1 time: 0.067501 May 24 10:18:05.519832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 221/466] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:05 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1238 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:06.348348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:06.348953 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:06.355102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:06.355550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:06.356313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:06.357083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:06.490906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:06.499164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fafec2ae-393d-48c4-a957-0d076c6cc9ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.155097 May 24 10:18:06.499739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 246/467] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:06.535675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:06.536289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:06.543613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:06.544071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:06.544844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:06.545300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:06.606382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:06.616258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dff81547-f197-4897-9a64-a913ed4e8690 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1234 microversion: 2.1 time: 0.084273 May 24 10:18:06.616893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 222/468] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:06 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1234 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:07.517201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:07.517809 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:07.525526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:07.525999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:07.526810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:07.527252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:07.635497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:07.636293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:07.644145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:07.644586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:07.646141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:07.646581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:07.664409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:07.672639 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-baad8433-432d-43d0-b757-74bcff98dc56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.159230 May 24 10:18:07.673358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 247/469] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:07.711656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:07.719425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ea8f1100-4ee4-4cef-b759-c7a952ddf15e tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1308 microversion: 2.1 time: 0.087778 May 24 10:18:07.720004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 223/470] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:07 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1308 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:08.689927 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:08.690568 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:08.697137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:08.697614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:08.698413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:08.698839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:08.737728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:08.738316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:08.744410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:08.744861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:08.745659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:08.746094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:08.834405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:08.843619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bd9cb06f-3184-4b8a-b3ef-af25631f2f71 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.157682 May 24 10:18:08.845080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 248/471] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:08.986890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:08.995716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-61f4d8bd-e3e0-478f-a8b6-709fe72763a6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.261161 May 24 10:18:09.004446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 224/472] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:08 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:09.195742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-1f8066c0-624b-49b6-80d5-994ec51850d0 req-f3ee39f0-7ea4-491f-8762-c7e4483ec3be service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d6651fff-37e6-4e58-9f84-5e58f2579414", "tag": "313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:09.206760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-1f8066c0-624b-49b6-80d5-994ec51850d0 req-f3ee39f0-7ea4-491f-8762-c7e4483ec3be service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:09.207500 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-1f8066c0-624b-49b6-80d5-994ec51850d0 req-f3ee39f0-7ea4-491f-8762-c7e4483ec3be service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:09.231036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-1f8066c0-624b-49b6-80d5-994ec51850d0 req-f3ee39f0-7ea4-491f-8762-c7e4483ec3be service nova] Creating event network-changed:313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f for instance d6651fff-37e6-4e58-9f84-5e58f2579414 on hv-cinder-84311 May 24 10:18:09.236576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-1f8066c0-624b-49b6-80d5-994ec51850d0 req-f3ee39f0-7ea4-491f-8762-c7e4483ec3be service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045136 May 24 10:18:09.237497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 249/473] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:18:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:09.862407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:09.863142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:09.869883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:09.870359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:09.871122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:09.871551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:10.011626 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:10.013056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:10.021283 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:10.021977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:10.023079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:10.023671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:10.039354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:10.045026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cd9ea3b6-8e67-4797-96a5-e15117617bca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.186700 May 24 10:18:10.045637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 225/474] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:10.164091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:10.172178 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-519bb42c-7413-44a2-b389-3e6757b7eefe tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.164574 May 24 10:18:10.172746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 250/475] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:10 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:11.062477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:11.063059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:11.070077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:11.070559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:11.071324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:11.071761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:11.189823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:11.190429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:11.196854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:11.199268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:11.199268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:11.199268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:11.213292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:11.221471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2049795c-87dd-4dda-9aa7-d9feb443ea5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.162915 May 24 10:18:11.222127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 226/476] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:11.324761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:11.332792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-beadbc98-570f-42b6-b615-2d359364831a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.146409 May 24 10:18:11.333387 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 251/477] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:11 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:12.240305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:12.240911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:12.247359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:12.247802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:12.248559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:12.248988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:12.349958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:12.350564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:12.356901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:12.357383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:12.358155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:12.358584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:12.377975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:12.385911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-af10fb22-7424-43a9-872a-e36fd179156a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.150387 May 24 10:18:12.386581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 227/478] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:12.490430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:12.497978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2c93a568-030a-4e66-87c4-7996ff3e485b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.152579 May 24 10:18:12.509675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 252/479] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:12 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:13.403830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:13.404433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:13.410885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:13.411319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:13.412077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:13.412508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:13.517250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:13.517942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:13.524549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:13.525003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:13.525798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:13.526238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:13.547867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:13.557208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec0fbfe7-310e-4b53-bdaf-701df57ce51d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.157202 May 24 10:18:13.557817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 228/480] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:13.657963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:13.666262 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-107730d7-fbc7-49b4-9016-4473124405a1 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.152952 May 24 10:18:13.666835 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 253/481] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:13 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:14.576088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:14.576698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:14.584196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:14.584699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:14.585498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:14.585938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:14.680290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-e2aab414-b68f-4bc8-99d0-c1fe80dcce53 req-35f64534-b833-4712-be10-faca9e72a653 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d6651fff-37e6-4e58-9f84-5e58f2579414", "name": "network-vif-plugged", "status": "completed", "tag": "313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:14.690831 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-e2aab414-b68f-4bc8-99d0-c1fe80dcce53 req-35f64534-b833-4712-be10-faca9e72a653 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:14.691269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-e2aab414-b68f-4bc8-99d0-c1fe80dcce53 req-35f64534-b833-4712-be10-faca9e72a653 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:14.702333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-e2aab414-b68f-4bc8-99d0-c1fe80dcce53 req-35f64534-b833-4712-be10-faca9e72a653 service nova] Creating event network-vif-plugged:313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f for instance d6651fff-37e6-4e58-9f84-5e58f2579414 on hv-cinder-84311 May 24 10:18:14.710570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-e2aab414-b68f-4bc8-99d0-c1fe80dcce53 req-35f64534-b833-4712-be10-faca9e72a653 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034274 May 24 10:18:14.711331 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 254/482] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:18:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:14.719255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:14.719998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:14.731673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:14.732271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:14.733299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:14.733745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:14.744827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:14.753259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1db8fe33-dce4-4ce6-a261-ecb5451dc266 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.180988 May 24 10:18:14.753864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 229/483] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:14.877713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:14.885732 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-54b8298c-2462-44b2-99fd-fe2557f7ee40 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.171060 May 24 10:18:14.886372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 255/484] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:14 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:15.771433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:15.772037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:15.773920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=6 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:15.779204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:15.779652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:15.780414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:15.780847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:15.902735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:15.903449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:15.905301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:15.905867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:15.906013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell0:SELECT=6 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:15.911212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:15.911693 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:15.912470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:15.912908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:15.914477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-017a1f6d-35ee-4a10-9e81-84296d59e3ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1714 microversion: 2.1 time: 0.147658 May 24 10:18:15.915130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 230/485] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1714 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:16.030919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:16.032081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:16.039560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:16.040015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:16.040810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:16.041441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:16.051695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:16.059108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-acfd80d2-effe-4274-adff-0e3e5b744318 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1364 microversion: 2.1 time: 0.160185 May 24 10:18:16.059675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 256/486] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:15 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:16.124655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] HTTP exception thrown: Cannot 'detach_volume' instance 79059316-a25c-4dd7-954c-0f7a413203a1 while it is in vm_state rescued May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Request method failure captured: May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments/3bc1cf9a-17de-4332-b457-fa4e80a5fe7f HTTP/1.1 May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_Tzo8x-h618vQIK2AZLsdhbocu8prJbiBdNbF4qbpYG1T25CEKu0iXYE0uhR0ccC8E3yZqmtj1CZrNPnmvDbAL3DpOJ2ErogKclQ3hggHrFsUzfMzEWXvOyC-9cvzh23HWiH6chQJDjdzC48vA8e-TI3CS-ooI8UxDpF3wTWTNLUA May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:18:16.130522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: baac9b5e923747e6986720e3ac849056 May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:18:16.132157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Cannot 'detach_volume' instance 79059316-a25c-4dd7-954c-0f7a413203a1 while it is in vm_state rescued May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'server_id': '79059316-a25c-4dd7-954c-0f7a413203a1', 'id': '3bc1cf9a-17de-4332-b457-fa4e80a5fe7f'} May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 79059316-a25c-4dd7-954c-0f7a413203a1 in vm_state rescued. Cannot detach_volume while the instance is in this state. May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:18:16.132908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 79059316-a25c-4dd7-954c-0f7a413203a1 while it is in vm_state rescued May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 79059316-a25c-4dd7-954c-0f7a413203a1 while it is in vm_state rescued May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:18:16.134198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Returning 409 to user: Cannot 'detach_volume' instance 79059316-a25c-4dd7-954c-0f7a413203a1 while it is in vm_state rescued {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:18:16.135142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3d1ae8f9-a3ef-46c8-8b32-99458f8ce737 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments/3bc1cf9a-17de-4332-b457-fa4e80a5fe7f" status: 409 len: 152 microversion: 2.1 time: 0.101382 May 24 10:18:16.135142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 231/487] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:18:16 2022] DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments/3bc1cf9a-17de-4332-b457-fa4e80a5fe7f => generated 152 bytes in 102 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) May 24 10:18:16.138834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:16.139367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:16.145649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:16.146197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:16.146958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:16.147384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:16.366700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f93c2da5-fd05-429c-9407-65782821d01e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/action" status: 202 len: 0 microversion: 2.1 time: 0.231829 May 24 10:18:16.367278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 257/488] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:18:16 2022] POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/action => generated 0 bytes in 233 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:18:16.375700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:16.376311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:16.384442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:16.384898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:16.385709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:16.386144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:16.522127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:16.530428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4333e93e-91c8-4243-a180-5d57445aa93d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1722 microversion: 2.1 time: 0.158680 May 24 10:18:16.531002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 232/489] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:17.077317 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:17.078022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:17.085063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:17.085549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:17.086324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:17.086755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:17.223003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:17.230849 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9cb44e72-97d1-45dc-bf5d-50bb1c4907d6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1549 microversion: 2.1 time: 0.158247 May 24 10:18:17.231348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 258/490] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:17 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1549 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:17.247339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:17.247986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:17.254804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:17.255256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:17.256030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:17.257038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:17.377313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a5566a20-aca2-45e3-89e4-f5228bcf9329 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.133930 May 24 10:18:17.377917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 233/491] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:18:17 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414/os-interface => generated 284 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:17.548156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:17.548760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:17.554993 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:17.555436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:17.556194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:17.556625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:17.803106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:17.812011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-41219412-f3ba-4018-8332-7605ea9571db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1722 microversion: 2.1 time: 0.267486 May 24 10:18:17.812695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 259/492] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1722 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:18.664830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-3a518b1d-c56b-4caf-8677-297ee85fff1b req-3b86a1d0-6be8-4c28-b567-8a4bfd95ef70 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d6651fff-37e6-4e58-9f84-5e58f2579414", "tag": "313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:18.673615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-3a518b1d-c56b-4caf-8677-297ee85fff1b req-3b86a1d0-6be8-4c28-b567-8a4bfd95ef70 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:18.678591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-3a518b1d-c56b-4caf-8677-297ee85fff1b req-3b86a1d0-6be8-4c28-b567-8a4bfd95ef70 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:18.684946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-3a518b1d-c56b-4caf-8677-297ee85fff1b req-3b86a1d0-6be8-4c28-b567-8a4bfd95ef70 service nova] Creating event network-changed:313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f for instance d6651fff-37e6-4e58-9f84-5e58f2579414 on hv-cinder-84311 May 24 10:18:18.690191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-3a518b1d-c56b-4caf-8677-297ee85fff1b req-3b86a1d0-6be8-4c28-b567-8a4bfd95ef70 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030273 May 24 10:18:18.690791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 234/493] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:18:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:18:18.829169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:18.829908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:18.837300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:18.838008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:18.838853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:18.839289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:18.843619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:18.996627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:19.008104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0744e29a-ab29-427e-aabc-7a688fe6ce46 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1722 microversion: 2.1 time: 0.182719 May 24 10:18:19.009304 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 260/494] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1722 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:20.027193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:20.027889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:20.034141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:20.036495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:20.036495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:20.036495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:20.168713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:20.176953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-48d2ccec-4961-40d6-bf36-aa09d0c78eea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1728 microversion: 2.1 time: 0.155121 May 24 10:18:20.177639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 235/495] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1728 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:20.193303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:20.199760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:20.200974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:20.201468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:20.202233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:20.202670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:20.319392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3def960d-3e2c-44b8-b99a-a62865ad1729 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.129815 May 24 10:18:20.319989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 261/496] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:18:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-interface => generated 284 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:21.251021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-70af3c19-60a0-40f8-968d-b01d80a48988 req-2071261e-ee09-45d1-ac30-45aff0e4eed7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "79059316-a25c-4dd7-954c-0f7a413203a1", "tag": "6c9affb3-0667-4743-a73b-88352bae25e5"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:21.259873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-70af3c19-60a0-40f8-968d-b01d80a48988 req-2071261e-ee09-45d1-ac30-45aff0e4eed7 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:21.260522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-70af3c19-60a0-40f8-968d-b01d80a48988 req-2071261e-ee09-45d1-ac30-45aff0e4eed7 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:21.276953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-70af3c19-60a0-40f8-968d-b01d80a48988 req-2071261e-ee09-45d1-ac30-45aff0e4eed7 service nova] Creating event network-changed:6c9affb3-0667-4743-a73b-88352bae25e5 for instance 79059316-a25c-4dd7-954c-0f7a413203a1 on hv-cinder-84311 May 24 10:18:21.282141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-70af3c19-60a0-40f8-968d-b01d80a48988 req-2071261e-ee09-45d1-ac30-45aff0e4eed7 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034722 May 24 10:18:21.283034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 236/497] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:18:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:23.298179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-ed91c2de-76e2-4e01-bd9d-0711b3caa607 req-113060a9-8daf-475f-bd92-2a993463bb70 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "79059316-a25c-4dd7-954c-0f7a413203a1", "tag": "6c9affb3-0667-4743-a73b-88352bae25e5"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:23.307032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-ed91c2de-76e2-4e01-bd9d-0711b3caa607 req-113060a9-8daf-475f-bd92-2a993463bb70 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:23.307468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-ed91c2de-76e2-4e01-bd9d-0711b3caa607 req-113060a9-8daf-475f-bd92-2a993463bb70 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:23.318340 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-ed91c2de-76e2-4e01-bd9d-0711b3caa607 req-113060a9-8daf-475f-bd92-2a993463bb70 service nova] Creating event network-changed:6c9affb3-0667-4743-a73b-88352bae25e5 for instance 79059316-a25c-4dd7-954c-0f7a413203a1 on hv-cinder-84311 May 24 10:18:23.323540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-ed91c2de-76e2-4e01-bd9d-0711b3caa607 req-113060a9-8daf-475f-bd92-2a993463bb70 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030011 May 24 10:18:23.324117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 262/498] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:18:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:45.204933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:18:45.205574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:45.207964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=37 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:45.208608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=91 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:45.214465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:45.214910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:45.215673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:45.216110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:45.237940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lazy-loading 'flavor' on Instance uuid 426e8866-95a1-42e7-9e88-29f93c5e7d7b {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:18:45.520650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-907a0d83-b039-4a43-b3a6-6b0ecda7858d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "POST /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b/action" status: 202 len: 0 microversion: 2.1 time: 0.319836 May 24 10:18:45.521285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 237/499] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:18:45 2022] POST /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b/action => generated 0 bytes in 321 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:18:45.530179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:45.536125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:45.536125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:45.536125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=10 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:45.538864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:45.539310 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:45.540080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:45.540514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:45.868424 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:45.875969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bba7101a-5a74-4a34-96a1-b0424f41875b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.349993 May 24 10:18:45.876529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 263/500] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:46.265972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:18:46.266480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:18:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:18:46.894412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:46.895109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:46.902130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:46.902579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:46.903341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:46.903776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:47.091542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:47.098998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-48a45587-3a6e-44a7-a4a0-154c3dbccb5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.209247 May 24 10:18:47.099565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 238/501] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:48.116504 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:48.117167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:48.123620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:48.124070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:48.124840 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:48.125292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:48.259161 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:48.266680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d2c53d48-8b11-42cc-843c-269799a3b560 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154338 May 24 10:18:48.267247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 264/502] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:49.283774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:49.284385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:49.291906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:49.292357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:49.293118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:49.293586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:49.422745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:49.430581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ecbfcb1b-26d1-4f87-8e5a-fba40bee6210 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150660 May 24 10:18:49.431141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 239/503] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:50.448162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:50.448771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:50.455268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:50.461719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:50.461719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:50.461719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:50.591724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:50.599909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-373158a1-336e-4ef6-b5db-d058a070b1bd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155034 May 24 10:18:50.600476 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 265/504] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:51.616488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:51.617100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:51.623770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:51.624273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:51.625060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:51.625525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:51.752779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:51.760526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9aba7a2e-6fb7-40a6-b88e-90dc6ebf5404 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148556 May 24 10:18:51.761089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 240/505] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:52.777981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:52.778501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:52.785038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:52.785514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:52.786280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:52.786750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:52.915332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:52.923299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ed0cb2d5-d7d9-47c3-8bed-0c2297df500c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149214 May 24 10:18:52.924051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 266/506] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:53.690121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:53.690623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:53.696923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:53.697399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:53.698151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:53.698579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:53.779796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lazy-loading 'services' on Instance uuid 79059316-a25c-4dd7-954c-0f7a413203a1 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:18:53.942235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:53.942235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:53.948761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:53.949460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:53.950240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:53.950680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:54.042521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e3f2460d-2eb5-488b-9509-4b2c9f982cfd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments/3bc1cf9a-17de-4332-b457-fa4e80a5fe7f" status: 202 len: 0 microversion: 2.1 time: 0.356040 May 24 10:18:54.043156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 241/507] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:18:53 2022] DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments/3bc1cf9a-17de-4332-b457-fa4e80a5fe7f => generated 0 bytes in 358 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:18:54.052126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:54.054047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:54.060336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:54.060905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:54.061719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:54.062151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:54.082378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:54.090308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cef42f44-a575-4dcb-8ba3-9ca9395fdcc1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153695 May 24 10:18:54.090895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 267/508] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:54.092608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ca950cde-86e1-4359-97ea-38124fa3b142 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044865 May 24 10:18:54.093166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 242/509] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:18:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:18:54.921280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:54.921774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:54.928013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:54.928451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:54.929202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:54.929694 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:54.952675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7d49a357-ded1-48af-9f06-613c797d64c5 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414/ips" status: 200 len: 144 microversion: 2.1 time: 0.092554 May 24 10:18:54.958358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 268/510] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:18:54 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414/ips => generated 144 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:55.103613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:55.104538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:55.107705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:55.108367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:55.111101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:55.111552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:55.112322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:55.112758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:55.114291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:55.115311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:55.116231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:55.116609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:55.147733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d23e062b-00ad-44db-a20e-f00ceb441931 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048342 May 24 10:18:55.148409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 243/511] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:18:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:55.254442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:55.263136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-756add75-5208-4ad1-b7a9-0d9a3f39a306 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.159247 May 24 10:18:55.263698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 269/512] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:18:56.159647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:56.165824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:56.167662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:56.168109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:56.168875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:56.169338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:56.200572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bf2145c8-45cc-4b2a-ae9e-8764865c967a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045004 May 24 10:18:56.201134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 244/513] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:18:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:56.280137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:56.280741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:56.287970 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:56.288412 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:56.289173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:56.289681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:56.423367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:56.431827 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-eca1efb9-2c7a-458a-afdc-413ec8fa77df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155622 May 24 10:18:56.432396 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 270/514] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:57.212712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:57.213642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:57.220039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:57.220479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:57.221229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:57.221685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:57.249521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0db8b7e0-5f23-4ed5-a536-14a054934de9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041309 May 24 10:18:57.250066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 245/515] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:18:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:57.448507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:57.449118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:57.455715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:57.456160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:57.456923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:57.457384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:57.593009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:57.601120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0f1753f8-16cc-4a3d-b2d5-6e4926f0b04c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156529 May 24 10:18:57.601733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 271/516] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:57 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:58.261948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:58.262858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:58.269543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:58.269988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:58.270742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:58.271715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:58.300962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5e170c70-e625-49fb-8708-2b9a0b83dc9d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042824 May 24 10:18:58.301502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 246/517] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:18:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:58.617814 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:58.618423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:58.625072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:58.625553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:58.626318 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:58.626751 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:58.762202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:58.769777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23d4748e-d550-435e-ac2c-5ba03c3e32af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155885 May 24 10:18:58.773186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 272/518] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:58 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:18:59.312173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:59.313011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:59.319870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:59.320311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:59.321076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:59.321541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:59.350095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ccd7f408-7818-495d-b279-6e189bd13c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041949 May 24 10:18:59.350649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 247/519] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:18:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:18:59.788009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:18:59.788611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:18:59.795062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:59.795499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:59.796253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:18:59.796680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:18:59.924046 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:18:59.931719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b3f3d137-4c2f-4971-b0e9-26b421ec1898 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148180 May 24 10:18:59.932402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 273/520] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:18:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:00.361828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:00.362768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:00.369871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:00.375344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:00.375344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:00.375344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:00.400642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-86089835-5f19-45de-ac83-c9756704edf1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042904 May 24 10:19:00.401225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 248/521] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:00.948756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:00.949404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:00.956037 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:00.956486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:00.957240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:00.957695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:01.089598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:01.097128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-839ce7c7-e3e2-4be9-bd92-5f6be02486be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152244 May 24 10:19:01.097708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 274/522] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:19:01.413001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:01.413861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:01.420308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:01.420755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:01.421536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:01.421962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:01.450604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cff0fe4d-72d0-4e1b-a289-20e5caa0220a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042366 May 24 10:19:01.451142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 249/523] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:02.115837 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:02.116479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:02.122677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:02.123124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:02.123888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:02.124321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:02.256050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:02.263469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d4957bf9-a444-4d18-9c74-6f0614630509 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152258 May 24 10:19:02.264032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 275/524] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:02.461906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:02.462828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:02.469366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:02.469813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:02.470572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:02.471001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:02.496555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:02.501058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7472a00e-1987-4eeb-b412-d806ab93aa82 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043295 May 24 10:19:02.501729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 250/525] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:02.809775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-48353c1f-e515-4fb2-9552-e8fee18665ea req-dcf1d785-66f1-40fc-a049-d269ffaefcbb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "426e8866-95a1-42e7-9e88-29f93c5e7d7b", "name": "network-vif-unplugged", "status": "completed", "tag": "0ed9cac0-6315-452a-9289-41b7a7958b3c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:19:02.817912 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-48353c1f-e515-4fb2-9552-e8fee18665ea req-dcf1d785-66f1-40fc-a049-d269ffaefcbb service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:02.818350 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-48353c1f-e515-4fb2-9552-e8fee18665ea req-dcf1d785-66f1-40fc-a049-d269ffaefcbb service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:02.829182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-48353c1f-e515-4fb2-9552-e8fee18665ea req-dcf1d785-66f1-40fc-a049-d269ffaefcbb service nova] Creating event network-vif-unplugged:0ed9cac0-6315-452a-9289-41b7a7958b3c for instance 426e8866-95a1-42e7-9e88-29f93c5e7d7b on hv-cinder-84311 May 24 10:19:02.835062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-48353c1f-e515-4fb2-9552-e8fee18665ea req-dcf1d785-66f1-40fc-a049-d269ffaefcbb service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.082110 May 24 10:19:02.835653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 276/526] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:19:02 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:03.281186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:03.281818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:03.288576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:03.289020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:03.289812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:03.290247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:03.423243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:03.431403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dbaec0b8-f793-4131-9d86-7a34cf127057 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154069 May 24 10:19:03.431963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 251/527] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:03.513546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:03.514542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:03.520678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:03.521171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:03.521958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:03.522385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:03.550532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-879ca525-f309-48e6-aa9b-972bbfb6c9a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041700 May 24 10:19:03.551070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 277/528] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:04.448775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:04.449397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:04.455803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:04.456242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:04.456990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:04.457458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:04.562798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:04.563800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:04.571298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:04.571832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:04.572673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:04.573178 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:04.588612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:04.597253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b57c4c3d-52cf-4760-8f41-47e1d6d57e76 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152263 May 24 10:19:04.597975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 252/529] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:04.605474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-34d4f333-36ee-4578-92c0-b002b0d2f0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046751 May 24 10:19:04.606181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 278/530] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:05.614488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:05.615152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:05.616258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:05.617187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:05.621795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:05.622239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:05.623001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:05.623458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:05.623731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:05.624205 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:05.625072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:05.625527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:05.655546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ed310765-7fc9-424f-89e1-fd83f1978065 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043493 May 24 10:19:05.656246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 279/531] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:05.752184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:05.759833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-334c9657-4037-45fb-98b6-aaf44ae7860c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149277 May 24 10:19:05.760481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 253/532] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:06.667596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:06.668508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:06.675297 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:06.675743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:06.676494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:06.676925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:06.694429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:06.706737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9a7d88f1-b9d7-405d-a9cb-528ee8c5f50d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043067 May 24 10:19:06.707286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 280/533] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:06.776938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:06.777500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:06.783985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:06.784461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:06.785213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:06.785662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:06.917414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:06.925154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-269acf94-e634-4f69-9b3f-cb57537ea1d6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152097 May 24 10:19:06.925753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 254/534] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:07.718512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:07.719428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:07.726345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:07.726799 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:07.727584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:07.728016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:07.756904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1f5b04a0-c771-4bce-b32e-f2db653539d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042476 May 24 10:19:07.757465 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 281/535] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:07.942194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:07.943444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:07.949971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:07.950411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:07.951159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:07.951588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:08.078798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:08.086725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a20524a-271b-4605-86d6-13a6d64a17ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148371 May 24 10:19:08.087295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 255/536] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:08.767723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:08.768683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:08.775660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:08.776107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:08.777030 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:08.777416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:08.807127 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fb163ce2-0438-48b2-a302-82eeca96126d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043481 May 24 10:19:08.807674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 282/537] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:09.104284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:09.104819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:09.110917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:09.111372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:09.112141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:09.112582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:09.244492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:09.252078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4096a358-fe9d-4262-9b51-2557efd61ed7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151733 May 24 10:19:09.252645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 256/538] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:09.819379 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:09.820297 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:09.826791 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:09.827180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:09.828070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:09.828625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:09.857770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6aa6b87e-932b-4e2b-9c34-0653d2d0d60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042934 May 24 10:19:09.858314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 283/539] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:10.269571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:10.270189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:10.277288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:10.277741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:10.278500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:10.278932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:10.409009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:10.416541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ce3db11f-12b4-46a2-baf7-844e03ef97d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151073 May 24 10:19:10.417047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 257/540] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:10.869195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:10.870160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:10.876584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:10.877034 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:10.877830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:10.878269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:10.906985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e4801a70-ce95-416a-9913-425b2106842e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041719 May 24 10:19:10.907532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 284/541] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:11.433424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:11.433941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:11.440474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:11.440924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:11.441706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:11.442140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:11.570150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:11.577591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ca77873a-f6a1-44af-9dfd-2b8ee1ea94e2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148106 May 24 10:19:11.578158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 258/542] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:11.918951 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:11.919869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:11.926609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:11.927112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:11.928048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:11.928484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:11.957226 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d209058d-b90d-49fa-90be-8b505537f332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042179 May 24 10:19:11.957738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 285/543] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:12.595427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:12.596033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:12.602924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:12.603411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:12.604244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:12.604677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:12.738337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:12.746410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f283fd4e-69c2-4907-b0eb-32d4e64fd214 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154975 May 24 10:19:12.746977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 259/544] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:12.969073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:12.970022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:12.977044 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:12.977500 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:12.978259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:12.978698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:13.007330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e08c8668-fefc-4be9-9bdf-010382b00b5e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042164 May 24 10:19:13.007876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 286/545] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:13.763489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:13.764026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:13.771143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:13.771749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:13.772534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:13.772968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:13.899411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:13.907606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0d75f71f-00ad-47c4-9f03-2b38808d5c0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148028 May 24 10:19:13.908173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 260/546] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:14.018924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:14.019761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:14.026240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:14.026691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:14.027488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:14.027919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:14.056906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2f5ed9e2-d181-48e1-8b78-afcae03456ea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041981 May 24 10:19:14.057475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 287/547] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:14.925021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:14.925592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:14.932087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:14.932533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:14.933302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:14.933730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:15.064471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:15.068066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:15.068988 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:15.072779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0b621c0e-6df2-479a-86e8-748f8f4fcb06 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151940 May 24 10:19:15.073384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 261/548] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:15.075460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:15.075910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:15.076673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:15.077099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:15.105950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8473faf3-58ee-4055-8998-bc10ee88b92f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042224 May 24 10:19:15.106495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 288/549] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:16.089741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:16.090365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:16.092637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=51 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:16.097388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:16.097845 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:16.098959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:16.099411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:16.118019 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:16.118940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:16.121295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=53 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:16.126097 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:16.126575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:16.127346 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:16.127785 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:16.158959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-68814383-3633-4836-82ef-d3ef8a9347e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045588 May 24 10:19:16.163270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 289/550] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:16.230647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:16.238188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-98339262-ddaf-44a3-8af3-e2677ea6eedc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152294 May 24 10:19:16.238753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 262/551] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:17.171448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:17.172359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:17.179797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:17.180241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:17.180998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:17.181478 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:17.210959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-18ca88f2-8f8b-4afb-ac66-1b82ab71ff7f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043827 May 24 10:19:17.211519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 290/552] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:17.255713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:17.256319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:17.262859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:17.263298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:17.264071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:17.264497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:17.394136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:17.401852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0edb3fa-2a56-4bd7-85cd-53340419b30e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149927 May 24 10:19:17.402429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 263/553] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:18.222455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:18.223385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:18.229949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:18.230396 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:18.231154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:18.231588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:18.261567 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-27c3c3f9-9138-41c1-a769-089c3233f944 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043050 May 24 10:19:18.262129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 291/554] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:18.419377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:18.419926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:18.426630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:18.427700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:18.428441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:18.428869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:18.557435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:18.565011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b7ee372f-f3f4-438c-9f6d-305a6db89147 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149502 May 24 10:19:18.565952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 264/555] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:19.273570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:19.274583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:19.281246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:19.281722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:19.282515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:19.282949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:19.312975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d10e7fd8-a99d-48e5-9608-d3f08f9c7634 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043550 May 24 10:19:19.313579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 292/556] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:19.582683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:19.583435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:19.589969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:19.590419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:19.591181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:19.591617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:19.719055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:19.727151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-80fded45-5a4a-4c49-afbb-8e56d49e3f4a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148365 May 24 10:19:19.727724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 265/557] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:20.325027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:20.326005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:20.332571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:20.333684 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:20.334535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:20.334969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:20.364349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aaf0971c-d869-424b-b456-21e4b18ad048 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043372 May 24 10:19:20.364904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 293/558] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:20.745080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:20.745709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:20.752377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:20.752827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:20.753675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:20.754114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:20.887048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:20.894445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8bace830-45a7-4967-89fc-0329b970b097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153252 May 24 10:19:20.895013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 266/559] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:21.376129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:21.377039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:21.383798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:21.384250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:21.385027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:21.385493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:21.415069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0d3c6bbd-7b4e-4c78-9bc2-17c358392a66 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043000 May 24 10:19:21.415674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 294/560] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:21.911904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:21.912540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:21.919280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:21.919722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:21.920478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:21.920902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:21.938717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:22.048128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:22.056354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-57a5bd38-7ae1-4d86-a3e8-957f0c2c18ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148649 May 24 10:19:22.056917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 267/561] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:22.426898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:22.428391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:22.435018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:22.435462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:22.436245 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:22.436676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:22.464979 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-24c6c094-659a-42ad-82da-8f48cc4278f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042351 May 24 10:19:22.465564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 295/562] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:23.073491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:23.074095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:23.080501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:23.080949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:23.081738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:23.082169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:23.215039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:23.222466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-26f3f43b-dafb-4c94-b59e-4ead2c3c394c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152910 May 24 10:19:23.223046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 268/563] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:23.476688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:23.477736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:23.484400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:23.484839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:23.485626 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:23.486060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:23.516020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6dba2548-75f3-49ae-a61d-2a1076372b91 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043370 May 24 10:19:23.516583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 296/564] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:24.239211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:24.239838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:24.246967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:24.247412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:24.248171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:24.248608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:24.382650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:24.390081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f650ab0c-3e95-4947-8543-50d8488cd52a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154726 May 24 10:19:24.390638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 269/565] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:24.527985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:24.528926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:24.535409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:24.535848 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:24.536601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:24.537026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:24.541325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:24.567191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-58aa2732-0f70-44a3-9fb2-42c6caa22af9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043109 May 24 10:19:24.567748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 297/566] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:25.407309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:25.407913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:25.415145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:25.415643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:25.416353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:25.416785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:25.546499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:25.554242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6e4c45f5-7ba8-46d0-a319-495ce0a7164c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150792 May 24 10:19:25.554807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 270/567] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:25.579026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:25.579950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:25.587131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:25.587539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:25.588301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:25.588736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:25.617169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d4905d1a-8272-48c5-a063-f848a405bddb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042355 May 24 10:19:25.617746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 298/568] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:26.571694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:26.572291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:26.578733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:26.579176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:26.579932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:26.580358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:26.628249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:26.629206 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:26.636781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:26.637245 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:26.638084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:26.638526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:26.670321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3d017568-2688-4331-8a34-41569d17da6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046185 May 24 10:19:26.670908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 299/569] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:26.717786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:26.726613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-297853ef-ac02-430e-8923-024d35acd76b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.159160 May 24 10:19:26.727296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 271/570] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:27.682750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:27.683666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:27.690280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:27.690728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:27.691491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:27.691924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:27.719696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9e311e2e-a50f-4a95-bbed-a469c8537c40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040893 May 24 10:19:27.720244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 300/571] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:27.744549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:27.745161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:27.751888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:27.752343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:27.753107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:27.753573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:27.887437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:27.895129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-74f5427e-a3a0-4c42-a45c-d107cfeb6181 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155269 May 24 10:19:27.895699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 272/572] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:28.732026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:28.732940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:28.739521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:28.739966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:28.741532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:28.741969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:28.770861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9c5f065c-971b-498a-b2be-f145854dd95c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042869 May 24 10:19:28.771356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 301/573] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:28.911874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:28.913166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:28.919597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:28.920067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:28.920796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:28.921256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:29.051221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:29.058949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a97976ce-d8c1-48b8-bb86-864b0521f8f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150846 May 24 10:19:29.059514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 273/574] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:29.782509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:29.783422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:29.790620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:29.791066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:29.791819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:29.792249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:29.821184 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-92fb7a4f-f653-4b73-815e-269fe234f4af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042623 May 24 10:19:29.821775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 302/575] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:30.087313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:30.087313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:30.087313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:30.087313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:30.088137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:30.088137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:30.214589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:30.222386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-32bce06b-526d-4ff5-b4cd-43f2d487fd5c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149743 May 24 10:19:30.222992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 274/576] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:30.832451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:30.833347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:30.841241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:30.841712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:30.842476 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:30.842909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:30.872105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d4b56ea4-349c-4aef-8a86-fb04e7331bfc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043778 May 24 10:19:30.872656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 303/577] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:31.239507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:31.240113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:31.246798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:31.247246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:31.248005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:31.248437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:31.396237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:31.402780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2a4e6f4e-35d7-4d06-8079-d2c385e6e424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.167167 May 24 10:19:31.403346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 275/578] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:31.884422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:31.885370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:31.891921 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:31.892372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:31.893143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:31.893608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:31.924708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9f375042-bddd-4c74-b7d9-6324c8e5d8c8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044824 May 24 10:19:31.925300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 304/579] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:32.419927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:32.420468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:32.427209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:32.427667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:32.428437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:32.428982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:32.566392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:32.574482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d0e17dd4-c533-4896-8883-9412ba9833e1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158548 May 24 10:19:32.575045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 276/580] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:32.936218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:32.937173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:32.943292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:32.943742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:32.945108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:32.945577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:32.974608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7e3a9295-40fc-4db9-8416-25626ecf1f54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042461 May 24 10:19:32.975161 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 305/581] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:33.592368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:33.593040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:33.599568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:33.600007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:33.601795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:33.602363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:33.742286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:33.749957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-85d1f0ff-ac82-47fa-bf43-6a0ab9523022 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.161534 May 24 10:19:33.751424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 277/582] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:33.987270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:33.988438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:33.996698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:33.997142 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:33.997981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:33.998469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:34.027347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1ee432dd-2677-487e-905a-a0ce39333c73 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044993 May 24 10:19:34.027901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 306/583] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:34.767825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:34.768433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:34.775925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:34.776314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:34.777126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:34.777590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:35.038402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:35.039419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:35.045992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:35.046434 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:35.047200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:35.047628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:35.079140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5a8fa23b-4dc1-4496-a5e8-83daec8a84cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044778 May 24 10:19:35.079723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 307/584] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:35.118888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:35.126438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e7064f7b-82f1-4c02-918d-54610e44e239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.362492 May 24 10:19:35.127025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 278/585] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:36.091722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:36.092671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:36.099199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:36.099646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:36.100404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:36.100924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:36.129326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f23a0e5d-e36d-49dc-901a-f444d44b3afb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041592 May 24 10:19:36.129882 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 308/586] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:36.143793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:36.144402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:36.151525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:36.151976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:36.152743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:36.153176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:36.281445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:36.289092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eceda70a-f9f8-44bf-978e-3feb547f58c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149187 May 24 10:19:36.289692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 279/587] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:37.141750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:37.142710 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:37.150196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:37.150652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:37.151436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:37.151878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:37.182544 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2d722503-6b89-471f-a806-ae35010a723d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044819 May 24 10:19:37.184522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 309/588] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:37.306513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:37.307120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:37.313667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:37.314110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:37.314871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:37.315299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:37.446175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:37.453580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1e6f5428-c552-462b-bc90-2ffc1ae145d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151102 May 24 10:19:37.454150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 280/589] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:38.194911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:38.195944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:38.202556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:38.203001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:38.203761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:38.204193 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:38.232940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3a4642b3-3904-4887-a671-7dbd14588841 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042509 May 24 10:19:38.233527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 310/590] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:38.470654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:38.471266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:38.478089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:38.478534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:38.479294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:38.479723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:38.609994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:38.617940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bb3d2765-2bb4-44f0-9a2b-9221e7583a98 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151108 May 24 10:19:38.618608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 281/591] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:39.244269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:39.245215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:39.251696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:39.252137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:39.252895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:39.253351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:39.282719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-62ab5cea-ad13-4e78-8cf8-c9aa00ae3e7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042424 May 24 10:19:39.283266 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 311/592] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:39.635557 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:39.636168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:39.642258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:39.642709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:39.643468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:39.643902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:39.773864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:39.781305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-310189bd-6dd1-4a23-9bd3-2c7d3ca8f102 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150427 May 24 10:19:39.781864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 282/593] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:40.294913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:40.295834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:40.302516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:40.302959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:40.303806 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:40.304517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:40.332595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b3a20d25-4c5a-456c-b66e-aa88a7ff0dda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041706 May 24 10:19:40.333143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 312/594] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:40.798438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:40.799027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:40.806050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:40.806499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:40.807263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:40.807698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:40.937764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:40.945776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-00474c4b-f37a-49ca-83f9-e066283d8a15 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151501 May 24 10:19:40.946337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 283/595] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:41.344038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:41.345055 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:41.351956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:41.352410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:41.353192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:41.353668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:41.378232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:41.383095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7951fd33-0cfb-48e0-ae8e-940a99ef8623 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042974 May 24 10:19:41.383650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 313/596] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:41.963213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:41.963810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:41.970284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:41.970727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:41.971474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:41.971899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:41.975793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:42.098863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:42.106391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-10295409-206b-42bf-b464-6702279ee659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147638 May 24 10:19:42.106959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 284/597] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:42.395476 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:42.396301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:42.402972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:42.403412 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:42.404164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:42.404591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:42.434436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cb2df215-9fe7-4ad4-a277-557e33565b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043373 May 24 10:19:42.434995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 314/598] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:43.123440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:43.124063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:43.130479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:43.130919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:43.131670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:43.132188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:43.263685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:43.271330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c655cd4d-216a-4875-9299-bc435e1e8cc9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151720 May 24 10:19:43.271899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 285/599] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:43.445985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:43.446935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:43.453360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:43.453805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:43.454558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:43.454986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:43.486363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0f2685f9-c99f-4a08-8241-70e49eb23aca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044975 May 24 10:19:43.487765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 315/600] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:44.288879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:44.289514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:44.296045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:44.296491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:44.297254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:44.297710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:44.429879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:44.437411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-faa867a9-54fb-49e0-8c8c-5fd251220c21 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152480 May 24 10:19:44.437979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 286/601] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:44.498627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:44.499547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:44.506203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:44.506650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:44.508144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:44.508569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:44.537085 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e9df13a0-deb5-4b7c-a616-f08ba62b56b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042385 May 24 10:19:44.537696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 316/602] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:45.453860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:45.454477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:45.461592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:45.462039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:45.462810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:45.463273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:45.548947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:45.549926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:45.556653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:45.557099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:45.557896 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:45.558326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:45.588005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a08900a6-61c6-424d-83e2-6dd9157eaa33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043179 May 24 10:19:45.588583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 317/603] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:45.596585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:45.605215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9e3e2e43-930f-4beb-a07e-441b9ac42a95 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155164 May 24 10:19:45.605805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 287/604] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:46.599157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:46.600112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:46.606596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:46.607047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:46.607808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:46.608242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:46.622271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:46.623007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:46.625296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:46.629972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:46.630422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:46.631183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:46.631653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:46.637890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6329453e-1c72-46a0-bb4f-8ab96b75bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042511 May 24 10:19:46.638456 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 318/605] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:46.760364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:46.767822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-818faa3f-306e-4342-9695-586eb646b1d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149379 May 24 10:19:46.768396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 288/606] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:19:47.650238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:47.651162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:47.653519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=60 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:47.658231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:47.658676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:47.659444 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:47.659875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:47.689974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3a5a2fd1-2f71-4cd8-86e4-ddead0f0afbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044320 May 24 10:19:47.690529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 319/607] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:47.785357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:47.785971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:47.793076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:47.793615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:47.794376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:47.794806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:47.932013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:47.939533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a363788f-9d90-448c-8bd6-5864c912d8cd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158061 May 24 10:19:47.940102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 289/608] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:48.701964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:48.708857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:48.709539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:48.709984 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:48.710906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:48.711866 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:48.740216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-865a9ad7-66f2-4110-b22c-6de02c467a33 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042882 May 24 10:19:48.740765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 320/609] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:48.957446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:48.958090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:48.965294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:48.965780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:48.966537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:48.966971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:49.098291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:49.106311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9151ec70-3885-48f4-b69d-f9b3917593ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152768 May 24 10:19:49.106873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 290/610] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:49.751713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:49.752642 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:49.759922 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:49.760365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:49.761116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:49.761580 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:49.789956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-24fb7d58-f86a-4d80-9ebc-354a8578c2f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042155 May 24 10:19:49.790500 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 321/611] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:50.122413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:50.122959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:50.129463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:50.129907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:50.130662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:50.131094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:50.262102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:50.269677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9cf17159-7db0-4ec8-8548-1f6f1e0096ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151196 May 24 10:19:50.270245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 291/612] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:50.801892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:50.802729 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:50.809385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:50.809865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:50.810628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:50.811064 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:50.839748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1703234d-d32b-47ed-ace9-a51d06874e98 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042294 May 24 10:19:50.840325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 322/613] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:51.287290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:51.287870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:51.295145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:51.295583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:51.296336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:51.296760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:51.436998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:51.444517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b184603c-74bc-4c77-b07a-11761c4ad169 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.161097 May 24 10:19:51.445174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 292/614] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:51.851313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:51.852230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:51.858804 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:51.859249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:51.860011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:51.860441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:51.889188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-661f35e6-0ae2-46b2-b33e-dd94f9c44b4b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041875 May 24 10:19:51.889770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 323/615] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:52.462120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:52.462727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:52.469140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:52.469618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:52.470384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:52.470818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:52.603453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:52.611099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef4d67f7-453c-4637-8697-d26a07d0daae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153457 May 24 10:19:52.611669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 293/616] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:52.900334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:52.901186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:52.907739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:52.908162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:52.908916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:52.909373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:52.938233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-30e14d1e-d605-406b-a6e0-f05cdce05232 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042149 May 24 10:19:52.938772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 324/617] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:53.627938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:53.628592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:53.635037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:53.635485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:53.636233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:53.636673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:53.764817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:53.772353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-caed0477-85b8-4444-ba73-07add4fc4a4e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148262 May 24 10:19:53.772943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 294/618] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:53.950409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:53.951333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:53.957798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:53.958239 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:53.958989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:53.959419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:53.987818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-379d0902-9d3e-4001-b046-c7a9867f642a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041376 May 24 10:19:53.988360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 325/619] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:54.790120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:54.790692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:54.797446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:54.798151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:54.799085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:54.799517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:54.930275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:54.937735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-717c6e86-68df-4695-b8ca-7182ae9b4ce0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151739 May 24 10:19:54.938297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 295/620] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:54.999310 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:55.000233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:55.006786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:55.007241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:55.008992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:55.009475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:55.038803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-29229f7f-0d08-444b-a054-8933233f3fd9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043480 May 24 10:19:55.039664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 326/621] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:55.954915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:55.955499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:55.962688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:55.963130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:55.963877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:55.964317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:56.050916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:56.051930 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:56.058593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:56.059046 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:56.059819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:56.060256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:56.089999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cc112cd6-f1d6-4589-8f45-51a33b7accf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043265 May 24 10:19:56.090572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 327/622] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:19:56.092189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:56.099756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e4e107fd-cc46-4b9d-9421-50c878d7bb55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148656 May 24 10:19:56.100312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 296/623] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:57.102080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:57.105380 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:57.109729 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:57.110172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:57.110922 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:57.111347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:57.117952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:57.118541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:57.124966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:57.125446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:57.126213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:57.126646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:57.142717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7156481f-287a-45db-9748-ddf71f0fb914 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044680 May 24 10:19:57.146687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 328/624] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:57.262063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:57.269409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-27c28ac4-7935-4958-b175-dd1f57995b9a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156112 May 24 10:19:57.272387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 297/625] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:57 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:19:58.154601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:58.155520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:58.162169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:58.162620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:58.163380 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:58.163955 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:58.193870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4339c53d-2ed9-4b9d-b3f3-799abc27e0e2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043184 May 24 10:19:58.194419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 329/626] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:58.285848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:58.286487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:58.292988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:58.293471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:58.294240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:58.294674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:58.424809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:58.432977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b8689275-823d-4cbc-940b-3c08cf49de5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150990 May 24 10:19:58.433578 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 298/627] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:58 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:19:59.206313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:59.213855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:59.215290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:59.215770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:59.216535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:59.216966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:59.233400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:59.245542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0b3cd918-dd95-4563-9f96-0097688cb7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043238 May 24 10:19:59.254204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 330/628] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:19:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:19:59.450416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:19:59.451205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:19:59.457812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:59.458265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:59.459031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:19:59.459478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:19:59.592916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:19:59.600573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fea214f2-6abc-41f8-8cc3-b0d46b771d81 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153985 May 24 10:19:59.601133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 299/629] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:19:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:00.256479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:00.257463 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:00.264445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:00.264946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:00.265843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:00.266281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:00.297605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c5175237-177b-4bdc-b1e0-58773490faad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045470 May 24 10:20:00.298420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 331/630] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:00.617499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:00.618271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:00.624752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:00.625194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:00.625990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:00.626417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:00.754108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:00.757529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:00.762662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-df172522-df66-46b0-af2f-32e9da62e06e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149025 May 24 10:20:00.763281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 300/631] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:01.310023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:01.310958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:01.317401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:01.317849 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:01.318600 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:01.319026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:01.347718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-901f107f-0809-4479-b86e-2000414f7cc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042341 May 24 10:20:01.348245 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 332/632] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:01.780264 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:01.780872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:01.787706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:01.788149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:01.788900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:01.789351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:01.918044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:01.925686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1556cfd0-dbde-40aa-b35b-c416877bb594 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149364 May 24 10:20:01.926254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 301/633] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:02.359361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:02.360300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:02.366347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:02.366790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:02.367541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:02.368042 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:02.396430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2311ffd6-d049-4408-9068-99151762cda8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041119 May 24 10:20:02.396975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 333/634] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:02.942502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:02.943109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:02.949707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:02.950152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:02.950906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:02.951622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:03.083726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:03.091335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-02c99110-b6a0-4839-ace5-463ddce99b3c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152734 May 24 10:20:03.091901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 302/635] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:03.408212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:03.409047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:03.415669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:03.416813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:03.417608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:03.418048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:03.446718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-635645f2-1fc5-4723-b4a4-c36ec1575136 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042568 May 24 10:20:03.447292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 334/636] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:04.108213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:04.108801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:04.115279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:04.115726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:04.116517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:04.116949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:04.250643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:04.258295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a78e9a14-2d4a-4d95-ae2e-830dca572b27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154038 May 24 10:20:04.258854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 303/637] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:04.458649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:04.459572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:04.466480 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:04.466932 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:04.467700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:04.468163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:04.497620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-834a38e6-728e-481e-b97d-f4b989ef53d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042349 May 24 10:20:04.498276 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 335/638] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:05.276466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:05.277406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:05.284646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:05.285085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:05.285889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:05.286310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:05.422517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:05.430289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-526e7722-e562-47af-8ff0-c8bd32b30505 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.157746 May 24 10:20:05.430863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 304/639] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:05.509013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:05.509957 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:05.516945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:05.517438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:05.518201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:05.518631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:05.547571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-73f14aa9-1fa2-4ab2-8bde-734281896289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042408 May 24 10:20:05.548418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 336/640] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:06.447722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:06.452666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:06.455984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:06.456481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:06.457355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:06.457820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:06.559869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:06.561120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:06.567566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:06.568023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:06.568800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:06.569241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:06.589759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:06.598012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6d9952ea-19dd-44fe-b935-3bb4e7e4e95b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153432 May 24 10:20:06.598012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 305/641] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:06.602587 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c7811e34-eeba-4580-bb76-7c4395e7406d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047156 May 24 10:20:06.603153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 337/642] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:07.614236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:07.615150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:07.615602 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:07.616517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:07.621947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:07.622390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:07.623245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:07.623672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:07.624012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:07.624569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:07.625479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:07.626008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:07.655174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5003700c-a76d-4793-a629-d7bd5c5c9550 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044873 May 24 10:20:07.655739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 306/643] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:07.761368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:07.769466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c63df089-616a-4ed9-beb1-e34fc8a9b74d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158822 May 24 10:20:07.770037 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 338/644] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:20:08.667071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:08.667981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:08.674820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:08.675265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:08.676023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:08.676448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:08.705376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d4a9db0b-962f-4616-a9c5-6c7c4324744d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042381 May 24 10:20:08.705940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 307/645] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:08.787033 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:08.787598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:08.793975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:08.795013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:08.795826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:08.796255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:08.925615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:08.933176 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-95c842f5-9812-4d09-9c77-7c4319e7e232 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150027 May 24 10:20:08.933760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 339/646] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:09.716032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:09.717047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:09.723632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:09.724075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:09.724833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:09.725277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:09.753498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c55b1c58-636c-4932-a384-e0a74d424847 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041341 May 24 10:20:09.754046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 308/647] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:09.949300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:09.949915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:09.956854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:09.957326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:09.958128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:09.958562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:10.091277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:10.098885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ae7e3974-f78d-4408-8c67-a76acad349e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153468 May 24 10:20:10.099490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 340/648] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:10.765945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:10.766869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:10.773298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:10.773751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:10.774508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:10.774944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:10.804031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ca94ae0e-ba91-414f-8243-ac1326aa77cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042979 May 24 10:20:10.804508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 309/649] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:11.116215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:11.116795 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:11.123989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:11.124430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:11.125185 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:11.125643 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:11.254757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:11.262237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e4363abb-c45c-4f74-a49a-c113ba54edd4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149980 May 24 10:20:11.262798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 341/650] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:11.816138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:11.817058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:11.823760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:11.824212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:11.824979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:11.825458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:11.854714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7224ae6b-9c95-440f-8e19-34de26feaaa9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042535 May 24 10:20:11.855202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 310/651] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:12.279881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:12.280485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:12.286963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:12.287410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:12.288174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:12.288605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:12.418212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:12.426889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cb5d82e5-e918-4359-9d7b-b1992e7c2656 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151067 May 24 10:20:12.427457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 342/652] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:12.866432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:12.867341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:12.873878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:12.874319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:12.875075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:12.875504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:12.905077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-97584242-0495-494b-8b58-13069b041baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041620 May 24 10:20:12.905655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 311/653] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:13.445314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:13.445931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:13.453129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:13.453717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:13.454469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:13.454903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:13.587227 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:13.595109 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a4aee438-7efa-4675-899f-613c32283c32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153693 May 24 10:20:13.595676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 343/654] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:13.916661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:13.917596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:13.924823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:13.925259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:13.926036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:13.926460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:13.954912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ea675947-7d40-466a-b2d0-d5c52a058dd0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042214 May 24 10:20:13.955463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 312/655] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:14.612058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:14.612663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:14.619618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:14.620078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:14.620836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:14.621290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:14.754942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:14.762475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5590bec2-9437-4964-88f5-d423169f4532 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154282 May 24 10:20:14.763034 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 344/656] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:20:14.966424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:14.967420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:14.974267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:14.974717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:14.975483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:14.975915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:15.004648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a5deabc2-7f6b-4669-a0c8-a010c7b75a63 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042189 May 24 10:20:15.005205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 313/657] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:15.779972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:15.780662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:15.787552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:15.788000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:15.788761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:15.789192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:15.920247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:15.928475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b3348457-3b96-4df9-ac1d-583ea138f201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152468 May 24 10:20:15.929040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 345/658] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:16.016120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:16.017015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:16.023408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:16.023847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:16.024628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:16.025023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:16.053640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6b64b477-2325-44ae-9832-a63e80ece530 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041467 May 24 10:20:16.054144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 314/659] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:16.946186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:16.946794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:16.953920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:16.954368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:16.955126 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:16.955558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:17.065635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:17.066584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:17.068899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=54 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:17.073723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:17.074156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:17.074919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:17.075350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:17.084515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:17.091544 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-96b54b74-b912-4cb8-a175-942345c9a633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149251 May 24 10:20:17.092355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 346/660] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:17.105815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-14652857-3135-4d86-9cf5-ca67d6d61ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044709 May 24 10:20:17.106364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 315/661] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:18.108927 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:18.109585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:18.111933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=56 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:18.116675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:18.127938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:18.128084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:18.128503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:18.128584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:18.128667 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:18.128667 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:18.128816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:18.128816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:18.128816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:18.128816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:18.156737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-acf0ebd3-2cb5-428b-9bbd-9e67c8fe7bbe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043353 May 24 10:20:18.157332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 316/662] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:18.250256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:18.257763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e4634778-3d56-4666-a760-624ac01476fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153238 May 24 10:20:18.258323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 347/663] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:19.167956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:19.168872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:19.175766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:19.176207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:19.176969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:19.177434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:19.193646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:19.206503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-549376a8-c748-4022-ac37-8a672789eb69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042480 May 24 10:20:19.209601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 317/664] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:19.275069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:19.275677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:19.282206 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:19.282653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:19.283415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:19.283855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:19.413971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:19.421665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c115657d-35f2-47bb-841d-b883f07fe825 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150322 May 24 10:20:19.422171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 348/665] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:20.219013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:20.220020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:20.226707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:20.227187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:20.227948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:20.228387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:20.258815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f9cc34a6-d979-45a1-bd3f-cbb817611194 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044652 May 24 10:20:20.259393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 318/666] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:20.438290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:20.438904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:20.445467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:20.445920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:20.446682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:20.447122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:20.578914 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:20.586399 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fbe07c1c-a7a6-462b-8fc4-f1d8003ac6f3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152055 May 24 10:20:20.586964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 349/667] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:21.271832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:21.272735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:21.279201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:21.279645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:21.281205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:21.281664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:21.309715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-62dd0c28-c6a8-4c6f-971b-77fae4562233 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041781 May 24 10:20:21.310257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 319/668] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:21.603876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:21.604481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:21.611133 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:21.611577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:21.612333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:21.612765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:21.746091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:21.754211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d213c2d8-5d5e-4a41-9844-8588c6fe60b7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154179 May 24 10:20:21.754783 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 350/669] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:22.320955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:22.321897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:22.329096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:22.329568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:22.330327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:22.330757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:22.358723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d8a47a58-3588-4bbe-ba1d-02c56c9c240c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041712 May 24 10:20:22.359266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 320/670] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:22.771146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:22.771748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:22.778185 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:22.778621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:22.779373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:22.779853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:22.906885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:22.914319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-933a1f69-2cec-470b-be20-c18d26b6d01f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147029 May 24 10:20:22.914889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 351/671] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:23.370456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:23.371378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:23.378947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:23.379394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:23.380156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:23.380584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:23.408694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a524cc65-30ec-4bf9-b6fc-a7ecc1f1ef6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042257 May 24 10:20:23.409246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 321/672] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:23.931776 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:23.932383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:23.939523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:23.939998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:23.940752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:23.941182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:24.071103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:24.079595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-227a14d0-3dac-4844-9cf8-1171ad8da524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151884 May 24 10:20:24.080165 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 352/673] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:24.420274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:24.421407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:24.428067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:24.428513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:24.429259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:24.429710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:24.458894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4b24f66e-c6dc-45db-a57c-c101bd00aceb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042589 May 24 10:20:24.459467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 322/674] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:25.195337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:25.195946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:25.201903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:25.202347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:25.203134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:25.203560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:25.333824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:25.341185 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cd9beba4-5c9e-4e7b-a3eb-46d3df8fa94e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.249474 May 24 10:20:25.341770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 353/675] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:25.470989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:25.472010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:25.479152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:25.479601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:25.480429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:25.480863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:25.510081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d500d1fb-6909-420d-af42-129580e3a25e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043685 May 24 10:20:25.510636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 323/676] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:26.358503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:26.359103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:26.366032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:26.366469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:26.367220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:26.367646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:26.496709 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:26.504766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-efbf7fc0-25b1-4f03-ab0f-d5a3d4e5e3c2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150135 May 24 10:20:26.505362 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 354/677] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:26.522068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:26.522990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:26.529388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:26.529840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:26.530602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:26.531116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:26.560658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-95446876-0971-41cd-80ff-cd45b2471871 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042436 May 24 10:20:26.561209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 324/678] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:27.521802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:27.522431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:27.528868 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:27.529339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:27.530093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:27.530523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:27.572465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:27.573414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:27.580635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:27.581079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:27.581877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:27.582314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:27.612436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e951ff0b-ef0c-464a-a6ce-2075382d68b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043866 May 24 10:20:27.613459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 325/679] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:27.663473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:27.670912 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-44bf95ce-7e68-4a4d-8a82-f2f77fe545d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153149 May 24 10:20:27.671471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 355/680] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:28.623601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:28.624520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:28.631861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:28.632314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:28.633087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:28.633554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:28.662664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd3b1d18-5900-4143-a879-e2c026328659 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043126 May 24 10:20:28.663228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 326/681] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:28.688510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:28.689757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:28.696238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:28.696678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:28.703189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:28.703189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:28.828329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:28.836004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-173391c9-f230-4c95-9e15-7e8c109b320e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151468 May 24 10:20:28.836575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 356/682] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:29.679963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:29.680875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:29.690333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:29.690834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:29.691627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:29.692056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:29.720317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-40529004-a1fa-4d8e-ac77-bb66862493aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044317 May 24 10:20:29.721608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 327/683] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:29.853045 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:29.853677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:29.860082 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:29.875509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:29.875509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:29.875509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:29.995662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:30.003245 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f35ec2de-374a-4530-b288-69f63059eaa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154123 May 24 10:20:30.003803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 357/684] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:20:30.732043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:30.732964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:30.740178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:30.740623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:30.741421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:30.741855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:30.770697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8cd05e24-a07d-4829-9664-50e785002b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042577 May 24 10:20:30.771284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 328/685] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:31.021365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:31.023471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:31.028666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:31.029110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:31.029914 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:31.030331 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:31.159331 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:31.166925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-77ddf9e7-856b-4848-8cff-4c5b0bf5bd94 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149523 May 24 10:20:31.167492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 358/686] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:31.782395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:31.783302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:31.789872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:31.790316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:31.791071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:31.791500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:31.821602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e9b69e17-d0be-40de-9861-8c11112cffdc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043134 May 24 10:20:31.822163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 329/687] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:32.184045 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:32.184690 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:32.190739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:32.191178 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:32.191941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:32.192369 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:32.321018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:32.328367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-de0837db-7620-4157-9751-55580dff49ef tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148317 May 24 10:20:32.328958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 359/688] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:32.834101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:32.834947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:32.841516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:32.841968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:32.842733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:32.843166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:32.872683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-09410c06-07ae-42a3-9a52-720b207d6365 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043342 May 24 10:20:32.873260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 330/689] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:33.347561 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:33.348174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:33.355099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:33.355546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:33.356315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:33.356783 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:33.498928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:33.506947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-026966cf-2a21-4277-b169-0cb152e91d48 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.164209 May 24 10:20:33.507563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 360/690] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:33.884674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:33.885530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:33.892101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:33.893158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:33.893952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:33.894379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:34.090874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a80e66bf-d43b-464c-948c-3c1d3a91ba23 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.210421 May 24 10:20:34.091423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 331/691] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:34.524275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:34.524870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:34.531321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:34.531742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:34.532492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:34.532920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:34.661685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:34.669027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-537d70ac-c8df-425f-8af4-cb0c5583188d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148618 May 24 10:20:34.669630 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 361/692] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:35.102826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:35.103770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:35.110553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:35.110889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:35.112451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:35.112883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:35.142125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a559a38a-468e-4250-bfe9-95489c0263b3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043206 May 24 10:20:35.142889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 332/693] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:35.686758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:35.687364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:35.694564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:35.695020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:35.695777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:35.696207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:35.831273 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:35.838777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b1c00f95-2e73-48aa-98c0-adb3f78f95de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155855 May 24 10:20:35.839349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 362/694] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:36.153865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:36.154699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:36.161260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:36.161726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:36.162486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:36.162916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:36.193198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b8404657-a119-4185-a968-591ccf5fd2ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043302 May 24 10:20:36.193774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 333/695] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:36.855833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:36.856354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:36.862900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:36.863341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:36.864102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:36.864531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:36.992006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:36.995372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:36.999632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b998ffa2-c2f2-41f1-bc6e-1e1291f27862 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147758 May 24 10:20:37.000202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 363/696] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:37.204125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:37.205958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:37.212722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:37.213177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:37.214004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:37.214442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:37.218054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:37.243744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-055279f3-12ee-4c3f-b04e-ae677533e1f3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043673 May 24 10:20:37.244554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 334/697] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:37.364750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:20:37.364954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:20:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:20:38.016535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:38.017128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:38.024029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:38.024505 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:38.025281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:38.025725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:38.160571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:38.168158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-701fba92-b403-41dc-96c1-fad85dd28e83 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155483 May 24 10:20:38.168725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 364/698] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:38.256601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:38.257586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:38.264084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:38.264534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:38.265314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:38.265753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:38.294360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4f54aefb-4f2a-4918-b745-7a15e0beb276 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042763 May 24 10:20:38.294833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 335/699] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:39.185855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:39.186466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:39.193027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:39.193510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:39.194268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:39.194701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:39.305907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:39.306883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:39.313026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:39.313505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:39.314269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:39.314702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:39.324940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:39.333284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a6a125c8-c77f-447f-a61b-048ae7a6c5d1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151215 May 24 10:20:39.333916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 365/700] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:39.344525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-115ecbdf-a771-4561-9f4c-bab91b7aa471 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042534 May 24 10:20:39.345065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 336/701] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:40.350884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:40.351528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:40.355771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:40.356705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:40.358373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:40.358819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:40.359579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:40.360008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:40.363902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:40.364346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:40.365105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:40.365569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:40.395349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7f544c00-5b9b-43cb-8300-04f4be4853ef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043401 May 24 10:20:40.395911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 337/702] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:40.490734 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:40.498441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9e10a247-35ce-4019-b9fd-7b67d9e6d584 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152117 May 24 10:20:40.499011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 366/703] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:41.407397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:41.408974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:41.416605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:41.417177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:41.418002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:41.418445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:41.450419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-25366a9f-e626-43ff-80d9-1336a5c66b0e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047374 May 24 10:20:41.451308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 338/704] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:41.515306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:41.515967 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:41.522893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:41.523343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:41.524110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:41.524550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:41.659918 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:41.667878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cdb1e429-64f3-4956-9207-dd16f03c3929 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156385 May 24 10:20:41.668377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 367/705] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:42.461986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:42.462892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:42.469286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:42.469725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:42.470477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:42.470904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:42.500466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d34abb81-3c95-468f-a99b-028a5ab84c68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042402 May 24 10:20:42.501029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 339/706] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:42.685774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:42.686533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:42.693945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:42.694394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:42.695154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:42.695588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:42.823947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:42.831560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-444af892-3e05-4deb-96f5-77ddf958d3b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149468 May 24 10:20:42.832120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 368/707] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:43.511237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:43.512153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:43.518658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:43.519637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:43.520398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:43.520833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:43.549704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8e64b245-9d91-4822-96f7-d72503eeb679 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042393 May 24 10:20:43.550352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 340/708] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:43.847919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:43.848527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:43.854972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:43.855417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:43.856173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:43.856601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:43.993830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:44.001495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ecb18185-6d4e-4693-b2d0-1b0a5fc5f14a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158010 May 24 10:20:44.002070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 369/709] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:44.561295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:44.562214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:44.569337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:44.569785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:44.570546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:44.570975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:44.599701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e4a4d5fe-28bd-468f-9225-425c02e629d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042342 May 24 10:20:44.600259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 341/710] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:45.020149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:45.020763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:45.027300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:45.027753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:45.028611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:45.028961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:45.159115 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:45.167513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aa51448d-fc6a-4345-ada0-a4e6b0281c97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152121 May 24 10:20:45.168091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 370/711] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:45.611481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:45.612357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:45.619259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:45.619741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:45.620511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:45.620948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:45.650857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-98b8cda9-ac8c-44a4-8af3-8eb7e88236e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043478 May 24 10:20:45.651397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 342/712] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:46.185131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:46.185800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:46.192867 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:46.193336 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:46.194096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:46.194528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:46.326637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:46.334742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3e410f87-37f6-476e-82cc-1ec7d2e91f79 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153462 May 24 10:20:46.335304 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 371/713] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:46.662024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:46.662952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:46.669413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:46.669863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:46.670627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:46.671063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:46.700420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c490d644-cc23-4802-8fd7-bcbe4f732ce6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042384 May 24 10:20:46.700966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 343/714] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:47.352224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:47.352865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:47.359364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:47.359811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:47.360575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:47.361010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:47.491352 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:47.498820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a28641b1-cbaa-4de9-86e5-a07b372107b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151080 May 24 10:20:47.499383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 372/715] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:47.712700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:47.713588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:47.720023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:47.720470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:47.721230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:47.721690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:47.749985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f454841b-443a-40c9-b242-cb9cc783215d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041866 May 24 10:20:47.750671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 344/716] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:48.515964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:48.516572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:48.518882 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=52 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:48.523892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:48.524333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:48.525086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:48.525541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:48.656020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:48.664177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2b58be5-d6b4-446b-9a60-3a7febc8c18f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152191 May 24 10:20:48.664753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 373/717] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:48.762400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:48.763313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:48.765628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=60 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:48.771239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:48.771620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:48.772383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:48.772809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:48.801810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8192e6db-8a8e-4084-a0e7-775a4b516d8b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043574 May 24 10:20:48.802360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 345/718] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:49.681691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:49.682311 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:49.688689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:49.689136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:49.689950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:49.690373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:49.813580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:49.814531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:49.822205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:49.822649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:49.823264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:49.823415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:49.823840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:49.830833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ee32988c-c8a0-4729-aaad-50f237f97c47 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153020 May 24 10:20:49.831398 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 374/719] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:49.853128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c4e166dd-7045-436f-96c2-9074c65e3356 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044059 May 24 10:20:49.853696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 346/720] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:50.848339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:50.848884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:50.855329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:50.855780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:50.856542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:50.856975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:50.865054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:50.865994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:50.872419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:50.872873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:50.873688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:50.874115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:50.902984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4decb6f-97e3-4193-ae8c-27b48bc7299c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042450 May 24 10:20:50.903599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 347/721] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:20:50.986031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:50.993614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2e586939-e7a6-4bf3-bc0d-d460724d20b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149712 May 24 10:20:50.994187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 375/722] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:51.914544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:51.915492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:51.922052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:51.922504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:51.923268 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:51.923959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:51.953134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-787a9e25-0726-4d4e-8699-d1b698d69503 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042540 May 24 10:20:51.953715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 348/723] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:52.010892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:52.011494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:52.018010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:52.018451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:52.019202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:52.019637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:52.151026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:52.158480 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-004625f5-a413-4984-b34a-ab71ffcdc0f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151389 May 24 10:20:52.159054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 376/724] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:52.964611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:52.965541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:52.971938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:52.972384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:52.973197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:52.973771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:53.002034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-87227bba-7755-477e-ab7c-68fb97322ab6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041373 May 24 10:20:53.002586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 349/725] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:53.177075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:53.178010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:53.184823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:53.185295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:53.186066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:53.186504 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:53.318471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:53.325937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-17325476-2c86-4106-addd-a44f9839f27e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153477 May 24 10:20:53.326507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 377/726] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:54.013762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:54.014677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:54.021657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:54.022103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:54.022861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:54.023286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:54.047446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:54.052046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5141be6f-f733-495e-aafa-d43cd5048b5b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042215 May 24 10:20:54.052661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 350/727] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:54.343356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:54.343872 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:54.351232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:54.351780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:54.352541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:54.352978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:54.481107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:54.488508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a4c61932-11ca-44fc-8e57-26c44aa950bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149140 May 24 10:20:54.489213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 378/728] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:55.064156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:55.065067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:55.071991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:55.072433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:55.073193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:55.073648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:55.103049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e7f96ba6-f456-4595-af03-04ab4fd450af tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043263 May 24 10:20:55.103910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 351/729] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:55.506181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:55.506721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:55.513163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:55.513639 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:55.514405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:55.514839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:55.649634 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:55.657102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-97a556e1-9b89-4405-a0c4-6f9337a724ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155504 May 24 10:20:55.657699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 379/730] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:56.114297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:56.115208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:56.121645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:56.122088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:56.122845 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:56.123271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:56.152971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-26d036be-ee73-4720-ba2a-4c6155684ed4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042559 May 24 10:20:56.153564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 352/731] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:56.675329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:56.675934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:56.683666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:56.684140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:56.685000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:56.685437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:56.705038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:56.822738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:56.830946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0c75ac4f-093e-42cf-95d4-393d5304c633 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.160480 May 24 10:20:56.831582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 380/732] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:57.167771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:57.169218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:57.179098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:57.179549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:57.180323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:57.180756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:57.212042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-04bc8cb7-2c1f-4f5a-b9c6-abdb24b607d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051214 May 24 10:20:57.216316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 353/733] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:57.850629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:57.851257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:57.858252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:57.858695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:57.859451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:57.859880 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:57.992808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:58.001080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d85e3fd2-1e5d-4b50-91f8-f83300f26044 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154969 May 24 10:20:58.001824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 381/734] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:57 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:58.226355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:58.227303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:58.233783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:58.234244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:58.235019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:58.235462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:58.264337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5e88b6ed-098a-46a1-8f03-435b29dd69c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042168 May 24 10:20:58.264888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 354/735] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:20:59.019283 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:59.019894 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:59.027169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:59.027616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:59.028382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:59.028815 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:59.164870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:20:59.174413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4397a883-33b2-4fa6-920e-65b090b4e37a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158605 May 24 10:20:59.175110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 382/736] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:20:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:20:59.276257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:20:59.277093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:20:59.283612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:59.284093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:59.284848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:20:59.285295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:20:59.314676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-60697ac1-86f3-476a-afa3-7328ba0ac8f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042597 May 24 10:20:59.315257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 355/737] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:20:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:00.192436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:00.193043 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:00.200096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:00.200574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:00.201418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:00.201891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:00.326902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:00.327841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:00.335174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:00.335647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:00.336420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:00.336847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:00.343897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:00.351758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-86fbaaf9-ad75-4f23-95f0-d96c35081ab7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.163159 May 24 10:21:00.352340 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 383/738] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:00.370216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-35f01ccc-eb8f-4ebd-afbe-702b67d3c4dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047311 May 24 10:21:00.370777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 356/739] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:01.369201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:01.369846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:01.376381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:01.376790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:01.377585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:01.378025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:01.382351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:01.383269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:01.389757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:01.390219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:01.390979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:01.391413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:01.421821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4a9e280a-e688-4315-a306-7ec504fc35d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043973 May 24 10:21:01.422309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 357/740] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:01.517150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:01.525222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7c6d5e36-dca9-424d-bd66-6be572374564 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.159874 May 24 10:21:01.525821 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 384/741] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:02.433647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:02.434623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:02.441185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:02.441666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:02.442436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:02.442869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:02.471281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1afaf18-a113-4fa5-8954-ad2e559da352 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041659 May 24 10:21:02.471842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 358/742] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:02.542787 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:02.543401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:02.549810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:02.550259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:02.551015 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:02.551530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:02.679194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:02.686692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5d66c917-200f-4aad-a877-861e3aaff0aa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147837 May 24 10:21:02.687469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 385/743] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:21:03.487415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:03.488336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:03.495307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:03.495758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:03.496559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:03.496993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:03.526658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a6c22bf4-da5a-4db5-bfaf-86bbdaf8c68e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044606 May 24 10:21:03.527209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 359/744] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:03.704173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:03.704772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:03.711399 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:03.711839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:03.712590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:03.713017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:03.851211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:03.858675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7d6ef55a-5aa3-4089-9db8-249c0250ac9d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158339 May 24 10:21:03.859231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 386/745] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:04.539363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:04.540283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:04.546825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:04.547275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:04.548043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:04.548482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:04.578698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd282bef-c5ae-4226-aedf-175dd2a10857 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044136 May 24 10:21:04.579265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 360/746] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:04.876039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:04.876589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:04.883787 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:04.886800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:04.886800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:04.886800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:05.028014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:05.035935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b5927bb2-e3e7-4968-8376-3b7f783d5432 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.163807 May 24 10:21:05.036651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 387/747] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:05.590324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:05.591247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:05.598134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:05.599117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:05.599880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:05.600316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:05.630139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8a6ffaed-aa83-4682-86a9-e4cc6c7248a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043754 May 24 10:21:05.630700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 361/748] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:06.054270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:06.054907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:06.061401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:06.061848 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:06.062606 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:06.063043 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:06.194352 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:06.201818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c1d6a9b1-f72d-4654-9ae0-6266904a2ea1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151536 May 24 10:21:06.202381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 388/749] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:06.641185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:06.642122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:06.649253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:06.649719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:06.650479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:06.650911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:06.680164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a5ddf04f-79c7-48e7-976f-765df2285d6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042946 May 24 10:21:06.680708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 362/750] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:07.218957 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:07.219558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:07.226280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:07.226727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:07.227474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:07.227898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:07.355664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:07.363259 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c3b5a1b9-cae5-4507-b8cc-b713d7424557 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148137 May 24 10:21:07.363852 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 389/751] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:07.692997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:07.693955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:07.700805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:07.707209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:07.707209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:07.707209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:07.731388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bdc1bf69-a4f6-41ff-a9a1-5472f6870f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043132 May 24 10:21:07.731943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 363/752] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:08.380169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:08.380791 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:08.387296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:08.387742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:08.388509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:08.388940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:08.519013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:08.527805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-76a13743-5cd1-43e2-b4c4-14b5d92b2135 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151718 May 24 10:21:08.528375 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 390/753] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:08.742532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:08.743444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:08.749652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:08.750102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:08.750856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:08.751281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:08.780342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-04a24f52-e3f6-411d-b124-0929f750c645 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041967 May 24 10:21:08.780916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 364/754] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:09.545343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:09.545960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:09.553447 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:09.553890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:09.554673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:09.555103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:09.792606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:09.793655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:09.800505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:09.800884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:09.801877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:09.802313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:09.833326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d6407170-102b-4a20-a14f-6b62e21a26b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044127 May 24 10:21:09.834200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 365/755] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:09.914190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:09.921638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2eb14264-781f-430b-b46f-4a9f8cb6f8c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.380217 May 24 10:21:09.922213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 391/756] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:10.845866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:10.846723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:10.854062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:10.854512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:10.855292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:10.855728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:10.884426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-69da9d6e-5078-4984-9834-f19af0640903 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042895 May 24 10:21:10.884982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 366/757] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:10.938933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:10.939546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:10.946356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:10.946808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:10.947578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:10.948017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:11.077750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:11.085233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8d107e4f-230e-47ed-a2af-e11bf379bbe7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150191 May 24 10:21:11.085823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 392/758] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:11.896889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:11.897760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:11.904149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:11.904593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:11.905370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:11.905800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:11.921935 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:11.934717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-de488fa3-608b-44b3-b646-a49ea024a3cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042600 May 24 10:21:11.935270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 367/759] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:12.101641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:12.102242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:12.109180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:12.109653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:12.110406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:12.110833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:12.251405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:12.258997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-925b6117-b5fd-43cb-bafd-0261302bb5e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.161377 May 24 10:21:12.259573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 393/760] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:12.947313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:12.948227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:12.955056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:12.955507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:12.956268 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:12.956700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:12.985148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c36c2c4c-0ef0-4da9-ada9-1668e1f7efaf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042232 May 24 10:21:12.985727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 368/761] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:13.275406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:13.275924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:13.282549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:13.283001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:13.283772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:13.284208 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:13.415437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:13.422997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cfe012a2-0b1a-4f97-87c0-798df85a497c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151514 May 24 10:21:13.423562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 394/762] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:13.997386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:13.998334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:14.005292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:14.005740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:14.006503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:14.006935 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:14.035547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d5203ab6-e6f5-4c48-ba8c-f9651e5a6908 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042160 May 24 10:21:14.036144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 369/763] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:14.440848 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:14.441482 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:14.448557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:14.448997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:14.449787 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:14.450218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:14.581151 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:14.588817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d06f7e83-b20c-487b-b56e-a7dd890a829e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151789 May 24 10:21:14.589414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 395/764] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:15.047482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:15.048436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:15.055493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:15.055944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:15.056712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:15.057152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:15.085982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0dda44a0-356b-4ca4-992a-b05a7592b706 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042536 May 24 10:21:15.086587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 370/765] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:15.607881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:15.608491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:15.615081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:15.615527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:15.616287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:15.616720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:15.744626 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:15.752057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a486fa8c-9f6c-490c-9d1d-2b56d297328d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147995 May 24 10:21:15.752656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 396/766] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:16.097946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:16.099393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:16.105860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:16.106341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:16.107170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:16.107612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:16.136602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0e90d56-28be-4a7c-8009-75f70ad6b7aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042601 May 24 10:21:16.137153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 371/767] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:16.769518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:16.770743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:16.777159 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:16.777651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:16.778428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:16.778867 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:16.782855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:16.908731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:16.916519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1a78f9cd-b4a3-4ec3-adce-ddd82844d521 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150901 May 24 10:21:16.917079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 397/768] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:17.148999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:17.149977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:17.156379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:17.156836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:17.157642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:17.158078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:17.187403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d34f6868-62de-4b16-80bc-479a2ba3c8fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043016 May 24 10:21:17.187957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 372/769] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:17.933437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:17.934041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:17.940467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:17.940941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:17.941718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:17.942153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:18.074934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:18.082968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d9f88369-089e-4344-bf92-54ab416eda97 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152918 May 24 10:21:18.083906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 398/770] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:18.198987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:18.199871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:18.206321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:18.206764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:18.208043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:18.208466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:18.237910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e48d4ccf-1edf-414c-b381-1a474dd91332 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042946 May 24 10:21:18.238463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 373/771] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:19.100970 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:19.101576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:19.103874 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=52 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:19.108737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:19.109180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:19.109966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:19.110398 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:19.243828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:19.248453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:19.249420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:19.251501 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e99b0731-44cb-4af7-a3ab-23a36349a782 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155048 May 24 10:21:19.252118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 399/772] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:19.256543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:19.257028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:19.257822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:19.258270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:19.288274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ad1812a1-acf6-40d3-9964-15838db8aa31 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043714 May 24 10:21:19.288843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 374/773] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:20.269277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:20.270117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:20.276727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:20.277170 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:20.277986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:20.278421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:20.301840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:20.309800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:20.309800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=60 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:20.309800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:20.310120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:20.310853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:20.311292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:20.341860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-71c2e15e-aa80-419a-b234-958393127a12 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046060 May 24 10:21:20.342434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 375/774] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:20.413415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:20.420981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c901df21-b6c6-4e21-aedf-69a525b7c71a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155557 May 24 10:21:20.421557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 400/775] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:21.352852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:21.353816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:21.360647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:21.361091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:21.361882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:21.362316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:21.390824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-acdd7a45-8715-43e9-b880-93a2afa35eac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041929 May 24 10:21:21.391375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 376/776] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:21.438831 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:21.439437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:21.446416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:21.446864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:21.447620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:21.448052 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:21.579141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:21.586780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7948d67a-ce13-40c6-bcbe-7713da182e69 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151770 May 24 10:21:21.587355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 401/777] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:22.402631 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:22.403544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:22.410077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:22.410532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:22.411280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:22.411709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:22.440741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e7d6b18c-8c46-44d0-966e-ff0632022c4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042058 May 24 10:21:22.441313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 377/778] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:22.603558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:22.604195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:22.610605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:22.611048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:22.611798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:22.612226 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:22.744104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:22.751700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-03e8d89b-0707-4b37-9aa8-d47d68af91ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151943 May 24 10:21:22.752265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 402/779] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:23.453041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:23.453989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:23.461426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:23.461885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:23.462717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:23.463245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:23.492492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e8e742d6-7000-47ab-bc21-464d98c4f7d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043530 May 24 10:21:23.493053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 378/780] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:23.769495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:23.771325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:23.777875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:23.778347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:23.779150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:23.779604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:23.913352 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:23.920965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d5c5faab-1861-43b2-8535-cf1cb662b8cb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155395 May 24 10:21:23.921564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 403/781] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:24.504045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:24.505146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:24.511567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:24.512002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:24.512759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:24.513189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:24.542380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bb268d7a-c226-4365-bc07-58857c2ed263 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042258 May 24 10:21:24.542940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 379/782] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:24.939075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:24.939770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:24.947791 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:24.948235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:24.949001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:24.949479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:25.079695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:25.088480 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e3db96df-122b-449e-a3a3-82848ea7e2ec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153301 May 24 10:21:25.089049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 404/783] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:25.554262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:25.555255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:25.561747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:25.562196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:25.562962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:25.563406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:25.591989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1ad13d2b-b88c-43c2-9a2a-1a7dea5bf4f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041737 May 24 10:21:25.592543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 380/784] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:26.105499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:26.106106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:26.113032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:26.113625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:26.114582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:26.115031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:26.249359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:26.257513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-162cbaa3-6bbb-4ccf-b888-276155132f4f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155860 May 24 10:21:26.258103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 405/785] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:26.603978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:26.604925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:26.611865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:26.612321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:26.613214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:26.614496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:26.643809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-06060424-49e9-4e9d-b0dd-cb4fd5acdb24 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043700 May 24 10:21:26.644617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 381/786] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:27.273834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:27.274435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:27.280811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:27.281254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:27.282032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:27.282462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:27.410460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:27.418229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-80987765-45a2-44d4-a0af-f9cbbcc25db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148662 May 24 10:21:27.418736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 406/787] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:27.655399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:27.656351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:27.662807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:27.663254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:27.664011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:27.664441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:27.694122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ac93753b-3195-450d-a9dc-ae93fc31fe61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042679 May 24 10:21:27.694671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 382/788] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:28.435948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:28.436555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:28.443252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:28.443702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:28.444457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:28.444887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:28.576825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:28.584862 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-beaaac25-a8bf-423c-9fd5-5ed334609982 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153054 May 24 10:21:28.585464 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 407/789] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:28.705581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:28.706418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:28.713500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:28.713953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:28.714726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:28.715153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:28.744777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aac225e0-5713-4079-a44c-eaea754f1be1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043282 May 24 10:21:28.745359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 383/790] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:29.602657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:29.603270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:29.609855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:29.610289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:29.611051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:29.611486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:29.739027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:29.746492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c93873d5-fc45-48a6-8c02-1abb0b8cde40 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148365 May 24 10:21:29.747138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 408/791] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:29.757019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:29.762013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:29.764329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:29.764781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:29.765580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:29.766017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:29.769801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:29.794968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-16610c13-4352-443f-ab77-7302ada48881 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042918 May 24 10:21:29.795527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 384/792] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:30.764324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:30.764950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:30.772438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:30.772896 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:30.773690 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:30.774129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:30.806646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:30.807561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:30.814170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:30.814614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:30.815395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:30.815817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:30.845860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9590de0c-88a6-46e9-994f-9f3ece672e26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043350 May 24 10:21:30.846354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 385/793] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:30.908629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:30.916362 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1122c2b9-894d-43db-bbbc-5d4361806b0e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155930 May 24 10:21:30.921464 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 409/794] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:31.858128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:31.859082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:31.865647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:31.866100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:31.866888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:31.867323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:31.896633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-62bf3466-3c67-48ee-8652-b66394054bf0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041825 May 24 10:21:31.897305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 386/795] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:31.933469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:31.934175 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:31.940573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:31.941013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:31.941795 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:31.942221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:32.073899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:32.081430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e8f68ff9-b71b-4749-9297-d79c94dabab1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151875 May 24 10:21:32.082004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 410/796] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:32.908868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:32.909736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:32.916870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:32.917399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:32.918159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:32.918588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:32.946551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e44563b0-091d-42c5-8bc6-3c12a0b30513 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041814 May 24 10:21:32.947098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 387/797] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:33.099579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:33.100187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:33.106871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:33.107237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:33.108004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:33.108439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:33.239348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:33.247180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3fd82877-719d-44a1-8e01-8593b289d85a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152172 May 24 10:21:33.247742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 411/798] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:33.958523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:33.959443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:33.966064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:33.966535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:33.967285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:33.967713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:33.997053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-22f414b6-277a-4c48-a9c2-80662e3104cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042666 May 24 10:21:33.997891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 388/799] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:34.264037 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:34.265617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:34.272436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:34.274348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:34.274348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:34.274348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:34.416184 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:34.423798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c3460b66-0675-4653-9316-c4c3f7a4237f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.163787 May 24 10:21:34.424620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 412/800] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:35.010437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:35.011354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:35.017818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:35.018247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:35.019007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:35.019435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:35.049925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7f6225a9-0622-4fad-a192-d39e4b848c11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044720 May 24 10:21:35.050489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 389/801] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:35.441457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:35.442069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:35.448691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:35.449137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:35.449945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:35.450381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:35.581981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:35.586247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:35.590623 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-25ae9c11-294d-45c4-8b31-b51fff349228 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153231 May 24 10:21:35.591197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 413/802] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:36.062898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:36.063877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:36.070948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:36.071402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:36.072155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:36.072583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:36.100551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd129b9f-12c6-4b55-9d4f-06eeed448203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042041 May 24 10:21:36.101092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 390/803] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:36.607944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:36.608552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:36.615861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:36.616305 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:36.617068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:36.617527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:36.745256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:36.752818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2985142d-351d-4536-ab6f-25253e9205b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149083 May 24 10:21:36.753404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 414/804] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:37.112178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:37.113227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:37.119906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:37.120353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:37.121118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:37.121744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:37.152252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d4075729-2214-4a98-bd09-2cff4e4243aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044029 May 24 10:21:37.152865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 391/805] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:37.770038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:37.770717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:37.777283 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:37.777779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:37.778542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:37.778973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:37.916117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:37.923610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-46d9a880-03a1-4c89-8633-bf6da2fcc318 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.157463 May 24 10:21:37.924182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 415/806] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:38.164698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:38.165999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:38.172115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:38.172590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:38.173373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:38.173806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:38.202244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-90d2de64-457d-4d19-8fde-b020b8f6d9f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042195 May 24 10:21:38.202794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 392/807] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:38.941220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:38.941860 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:38.948944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:38.949431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:38.950229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:38.950669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:39.081595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:39.089058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3544352d-fd62-41e2-afbb-9b4fc9593201 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151687 May 24 10:21:39.089648 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 416/808] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:39.213440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:39.214291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:39.220353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:39.220801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:39.221591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:39.222023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:39.250548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fb29a57e-942b-4038-a0c4-093b68064800 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041374 May 24 10:21:39.251128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 393/809] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:40.106007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:40.106646 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:40.113099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:40.113582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:40.114341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:40.114777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:40.246896 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:40.254393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9cd9ca0f-d8ce-4089-885d-18cbc1e09c03 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152335 May 24 10:21:40.254962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 417/810] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:40.268124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:40.269037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:40.276296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:40.279821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:40.279821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:40.279821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:40.307406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9121b3a1-9315-414d-88ea-a382db8e2387 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043180 May 24 10:21:40.307973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 394/811] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:41.272976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:41.273606 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:41.280111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:41.280552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:41.281323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:41.281751 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:41.318401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:41.319312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:41.325858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:41.326302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:41.327065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:41.327494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:41.357351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4f0bad21-d168-4c11-b046-895fa4859bae tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042905 May 24 10:21:41.357918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 395/812] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:41.412793 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:41.420429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ac7b71a4-4593-4279-ba0e-4169deb0a7a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151777 May 24 10:21:41.420934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 418/813] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:42.369839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:42.370760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:42.377223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:42.377705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:42.378477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:42.378916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:42.407824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-137adb8a-7a2c-496f-9029-2ed120a7bcef tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042270 May 24 10:21:42.408381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 396/814] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:42.437674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:42.438282 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:42.444935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:42.445530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:42.446289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:42.446722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:42.578381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:42.586016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-766a0bc6-c19f-4566-9fd4-c8a0a1c9d677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152183 May 24 10:21:42.586582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 419/815] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:43.420290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:43.421163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:43.428857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:43.429331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:43.430102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:43.430538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:43.462130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3c6ff20a-3415-4bc6-8130-0e25b2f3ead5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045858 May 24 10:21:43.462695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 397/816] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:43.602254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:43.602869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:43.609413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:43.609862 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:43.610625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:43.611059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:43.738939 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:43.747035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cacb8472-661d-42d9-a886-146b4b895229 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148898 May 24 10:21:43.747610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 420/817] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:44.473697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:44.475427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:44.482239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:44.482694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:44.483486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:44.483926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:44.515220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-64273749-f3a8-493a-bd29-46b262c060cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045499 May 24 10:21:44.515791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 398/818] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:44.764688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:44.765194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:44.772625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:44.773068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:44.773859 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:44.774288 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:44.903266 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:44.910744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7b94671b-5072-411e-9a08-433f4d146138 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150016 May 24 10:21:44.911348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 421/819] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:45.526906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:45.527826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:45.534295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:45.534744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:45.535508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:45.535943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:45.565910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-01fcb94e-d45e-4faa-a849-6aa7de7c1f4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042991 May 24 10:21:45.566459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 399/820] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:45.928168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:45.928719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:45.935292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:45.935720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:45.936471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:45.936897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:46.067633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:46.075478 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23edd312-868f-4d63-ac67-e26edae995b8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151671 May 24 10:21:46.076064 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 422/821] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:46.577949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:46.578866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:46.585672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:46.586125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:46.586902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:46.587334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:46.611645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:46.616578 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e019ee0a-cf3b-4cb4-8cc4-e818c70711a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042693 May 24 10:21:46.617130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 400/822] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:47.093439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:47.094051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:47.101155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:47.101628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:47.102394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:47.102826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:47.232612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:47.240107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-912bfbfb-5f74-4f07-b33c-67b3ec54fa58 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150530 May 24 10:21:47.240683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 423/823] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:47.628479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:47.629438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:47.636527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:47.636977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:47.637785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:47.638223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:47.667434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-be5d4479-3a20-4943-9b99-7ec5474d72e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043331 May 24 10:21:47.667989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 401/824] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:48.257018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:48.257771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:48.264339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:48.278371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:48.278371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:48.278371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:48.397107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:48.404731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6266ca79-18a8-4f3d-82b0-36912ddab3c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151517 May 24 10:21:48.405300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 424/825] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:48.679246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:48.680166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:48.686871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:48.687324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:48.688095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:48.688534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:48.719735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-58a84d74-74ce-43e8-a14b-ad4df268ef5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043579 May 24 10:21:48.719735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 402/826] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:49.422543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:49.423068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:49.425668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=52 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:49.430907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:49.431353 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:49.432118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:49.432552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:49.569384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:49.577514 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-51e483ef-3cb6-4441-afe0-2d97a810c3f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158879 May 24 10:21:49.578086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 425/827] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:49.730572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:49.731484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:49.737911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:49.738360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:49.739117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:49.739541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:49.768275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eba4c074-146a-4b4b-b1e7-2030ea80f23e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042191 May 24 10:21:49.768819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 403/828] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:50.594356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:50.594998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:50.601590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:50.602084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:50.602958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:50.603389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:50.732193 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:50.739781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-16dacd94-c565-41b0-8196-3531af9e9225 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149313 May 24 10:21:50.740350 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 426/829] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:50.780506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:50.781452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:50.787802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:50.788243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:50.788992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:50.789452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:50.818021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d83a926c-a251-4961-8b7b-3ce2c68f539a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041407 May 24 10:21:50.818563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 404/830] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:51.756824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:51.757456 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:51.763997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:51.764445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:51.765203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:51.765703 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:51.830255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:51.831178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:51.833500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=60 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:51.838870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:51.839326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:51.840101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:51.840535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:51.870661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a482226-61d0-4c18-b051-05d1f1184334 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044324 May 24 10:21:51.871237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 405/831] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:51.900456 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:51.907806 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2970a1d-2e68-4fd1-9bcb-62891578004a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154897 May 24 10:21:51.908364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 427/832] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:52.882112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:52.882949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:52.889733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:52.890224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:52.890980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:52.891408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:52.920328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2e0bb0c2-b2d0-4cff-8589-3cb028029446 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042269 May 24 10:21:52.920882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 406/833] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:21:52.925631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:52.926240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:52.933651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:52.934097 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:52.934889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:52.935322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:53.063257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:53.070659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e14f259d-06ad-4eac-b67c-c177907199e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149298 May 24 10:21:53.071251 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 428/834] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:21:53.932015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:53.932938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:53.939442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:53.939889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:53.940666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:53.941087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:53.971387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2aecd777-3279-48e7-a879-3e8e64dc5be4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043921 May 24 10:21:53.971941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 407/835] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:54.087512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:54.088127 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:54.094622 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:54.095069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:54.095832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:54.096267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:54.229335 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:54.236803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f8d2495f-fb76-48d1-9a82-41f1f0c02944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153121 May 24 10:21:54.237387 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 429/836] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:54.983050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:54.983967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:54.990694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:54.991141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:54.991898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:54.992333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:55.021615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b15d724d-5fa2-4a31-85bc-91f238318b13 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042611 May 24 10:21:55.022164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 408/837] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:55.254605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:55.255210 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:55.262555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:55.263000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:55.263765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:55.264200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:55.282540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:55.392925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:55.400449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-81074084-68e2-4b0a-8844-0595f99b9a14 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149797 May 24 10:21:55.401013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 430/838] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:56.033848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:56.034770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:56.041730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:56.042183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:56.042958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:56.043395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:56.073765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2ac755fe-1ce7-4641-86b4-97736d199769 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043840 May 24 10:21:56.074338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 409/839] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:56.417817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:56.418427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:56.425252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:56.425735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:56.426507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:56.426945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:56.562405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:56.569936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-00ab1454-fccb-420e-b112-88863d90c654 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156748 May 24 10:21:56.570512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 431/840] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:57.085079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:57.086048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:57.092529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:57.092978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:57.093781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:57.094219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:57.122976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd6e8471-50c5-403b-943b-1377fe3a633c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041912 May 24 10:21:57.123521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 410/841] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:57.588397 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:57.589007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:57.595613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:57.596081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:57.596842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:57.597290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:57.729428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:57.736963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-27b281f1-8c41-48a8-b081-dc3b4b062c10 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153153 May 24 10:21:57.737591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 432/842] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:57 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:58.134744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:58.136157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:58.143011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:58.143458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:58.144238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:58.144676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:58.174037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-171e599a-5b1a-41ce-82e9-1a3dbda575c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043251 May 24 10:21:58.174593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 411/843] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:58.754419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:58.755117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:58.762475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:58.762928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:58.763696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:58.764131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:58.896249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:21:58.904573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a6b4d12f-54b0-4f29-8cd0-bc348b695e3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154043 May 24 10:21:58.905150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 433/844] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:58 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:21:59.186656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:59.190569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:59.194129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:59.194582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:59.195344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:59.195778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:59.226243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3b338893-44c6-4f3e-bac4-1674b1a1f566 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044216 May 24 10:21:59.226699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 412/845] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:21:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:21:59.921902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:21:59.922532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:21:59.929202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:59.929675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:21:59.930428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:21:59.930883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:00.066461 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:00.074401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8c60c54a-564f-4c2f-97d9-85d1df4e15d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156100 May 24 10:22:00.074977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 434/846] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:21:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:00.222530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:22:00.222710 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:22:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:22:00.237737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:00.238670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:00.245405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:00.248082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:00.248082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:00.248082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:00.275946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9d76aaed-7470-469b-8caa-5f4fe8b9b40e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042231 May 24 10:22:00.282705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 413/847] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:01.091999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:01.092609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:01.099141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:01.099588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:01.100350 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:01.100785 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:01.229979 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:01.237500 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-96883baa-a52a-4e30-8835-63dca7d87893 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149957 May 24 10:22:01.238061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 435/848] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:01.288102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:01.289027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:01.295741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:01.296191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:01.296952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:01.297419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:01.326799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5280d3ec-d77b-4d94-ba8d-51f74af54344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042622 May 24 10:22:01.327356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 414/849] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:02.255071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:02.255682 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:02.261934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:02.262381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:02.263139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:02.263568 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:02.338456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:02.339635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:02.346136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:02.346585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:02.347349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:02.347786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:02.377345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ccb4ac8b-f671-4dc3-9c64-23868c0caf90 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043300 May 24 10:22:02.377910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 415/850] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:02.396048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:02.404018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ce7b86dc-5789-4a5a-a26b-676c1c07cc43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152824 May 24 10:22:02.404568 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 436/851] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:03.389079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:03.390043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:03.396497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:03.396944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:03.397748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:03.398184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:03.421048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:03.421595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:03.427873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5c7df94e-62c7-4d45-a968-e55d3a7b46c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042796 May 24 10:22:03.428393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:03.428582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 416/852] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:03.428836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:03.429627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:03.430059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:03.560155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:03.567725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7f1b98f3-ecde-4a97-bd25-a8a509936baa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151228 May 24 10:22:03.569013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 437/853] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:04.439527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:04.440515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:04.447049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:04.447498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:04.448264 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:04.448701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:04.466400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:04.478891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d262ca44-3957-4743-a72e-7cb6aca14037 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043313 May 24 10:22:04.479463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 417/854] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:04.583812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:04.584417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:04.591111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:04.591563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:04.592330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:04.592767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:04.721163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:04.728590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a11c0479-89e2-43f5-96c2-69a62e27c34e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148617 May 24 10:22:04.729154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 438/855] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:05.492912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:05.493876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:05.500728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:05.501185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:05.501974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:05.502412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:05.532125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e9dbb1d7-56c3-45d1-8a28-e7ff4fcf83dc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043453 May 24 10:22:05.532682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 418/856] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:05.745113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:05.745677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:05.752996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:05.753483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:05.754253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:05.754685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:05.884818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:05.893032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a99f7c30-f55a-4a2e-9864-770afe978811 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151814 May 24 10:22:05.893621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 439/857] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:06.274543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:22:06.274742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:22:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:22:06.544277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:06.545234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:06.552022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:06.552473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:06.553279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:06.553720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:06.583255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4085b66-9439-466e-b063-1625a08e7abc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043533 May 24 10:22:06.583817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 419/858] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:06.910120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:06.910727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:06.917186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:06.917657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:06.918644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:06.919083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:07.047408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:07.055373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-105cba6d-98e9-413d-916e-6a5b584a90dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149079 May 24 10:22:07.055937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 440/859] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:07.595345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:07.596289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:07.603055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:07.603505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:07.604277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:07.604711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:07.634558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-88e833fc-bec9-4576-95c5-3a5b8b4f16f1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043213 May 24 10:22:07.635125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 420/860] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:08.072258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:08.072892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:08.080947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:08.081456 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:08.082225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:08.082692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:08.214418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:08.221942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-344c2955-a056-4d70-a85d-cd8b996d1da1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153513 May 24 10:22:08.222506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 441/861] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:08.646490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:08.647464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:08.653995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:08.654437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:08.655195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:08.655627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:08.685210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-88de2861-babb-42b5-b0a5-49e3692f7fc3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042659 May 24 10:22:08.685788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 421/862] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:09.238265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:09.238907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:09.245382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:09.245830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:09.246794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:09.247233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:09.381248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:09.389561 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9aad9505-c3ed-4b2a-aef3-5ab59456a58c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154960 May 24 10:22:09.390160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 442/863] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:09.696459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:09.697416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:09.704973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:09.705465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:09.706236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:09.706675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:09.735457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6b3a6dfe-dd30-4537-98fb-179cc9800fcf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042999 May 24 10:22:09.736017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 422/864] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:09.877920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:22:09.878081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:22:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:22:10.406593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:10.407232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:10.413812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:10.414257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:10.415067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:10.415457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:10.551073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:10.558401 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ffe7a2ae-7de3-40ec-a384-eaa606e72c6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156255 May 24 10:22:10.559120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 443/865] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:10.747153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:10.748076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:10.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:10.755147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:10.755914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:10.756347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:10.786282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4c8189bf-bcdd-4b2b-bd3c-e97cbc3577a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043225 May 24 10:22:10.786836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 423/866] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:11.575376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:11.575966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:11.582596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:11.583047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:11.583809 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:11.584250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:11.712632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:11.721329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-efbe1384-db5e-4a57-9322-2ccb660324d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150158 May 24 10:22:11.721899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 444/867] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:11.797641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:11.798557 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:11.805494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:11.805943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:11.806700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:11.807133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:11.835223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4e987f70-d52d-4c89-9a3a-aacb22f93fe5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041969 May 24 10:22:11.835701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 424/868] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:12.739376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:12.739985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:12.747111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:12.747556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:12.748314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:12.748744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:12.846823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:12.847677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:12.854217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:12.854673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:12.855447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:12.856559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:12.879366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:12.886533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8d158030-3223-4571-8e53-79366b7423d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043838 May 24 10:22:12.886811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2650a344-3b0d-4426-a9c2-b7eac4e1656e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151251 May 24 10:22:12.887434 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 445/869] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:12.888356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 425/870] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:13.899861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:13.900719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:13.904578 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:13.905194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:13.908680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:13.909130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:13.909949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:13.910389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:13.911653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:13.912113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:13.912884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:13.913339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:13.940571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4af1b0d4-8aa7-4ddf-a486-d132d8b310f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044876 May 24 10:22:13.941143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 446/871] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:14.043329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:14.051053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e9cc8d1f-aa2a-407a-9dd5-a28577d958bb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151320 May 24 10:22:14.051622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 426/872] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:14.952411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:14.953352 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:14.959816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:14.960261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:14.961011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:14.961470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:14.965629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:14.990533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-055723ad-6e44-477e-a0bb-5a9517515558 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042372 May 24 10:22:14.991076 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 447/873] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:15.068160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:15.068774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:15.076050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:15.076501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:15.077284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:15.077719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:15.212639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:15.221252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-beeebf93-18f8-4290-9690-e3e590dd497a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.157264 May 24 10:22:15.221849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 427/874] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:16.003179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:16.004125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:16.010800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:16.011272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:16.012177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:16.012728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:16.040684 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0b5ce35a-728f-4456-878b-85eed874bf36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041774 May 24 10:22:16.041223 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 448/875] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:16.239183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:16.239827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:16.246427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:16.246883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:16.247632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:16.248067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:16.377183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:16.384596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5c1ebc4a-ecc6-426a-8e1f-74987d82c1e3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149274 May 24 10:22:16.385160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 428/876] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:17.053652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:17.054584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:17.061944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:17.062397 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:17.063160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:17.063597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:17.092814 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e1dc165c-2635-4697-9822-64c7f46ea77e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043399 May 24 10:22:17.093410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 449/877] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:17.400594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:17.401132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:17.407621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:17.408069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:17.408832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:17.409277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:17.535199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:17.542679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-89adbdae-bdb2-482e-b310-9f3fe242ad3e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.145990 May 24 10:22:17.543767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 429/878] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:18.104394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:18.105335 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:18.111898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:18.112355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:18.113116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:18.113585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:18.142512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-eae7198a-896f-4c64-922c-e78d288df7df tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042079 May 24 10:22:18.143083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 450/879] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:18.560847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:18.561481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:18.567976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:18.568424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:18.569215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:18.569672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:18.702995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:18.710569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-037545d1-99e8-4c31-b72e-47f2d6b4a12e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154139 May 24 10:22:18.711133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 430/880] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:19.154119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:19.155041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:19.161562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:19.162013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:19.162780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:19.163212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:19.193440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f3df7962-1bba-4bf9-87bf-3cbedb6b91a3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043306 May 24 10:22:19.194001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 451/881] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:19.727674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:19.728285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:19.735242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:19.735691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:19.736450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:19.736887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:19.863865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:19.871349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-33fd0a61-1810-4437-9c84-8e51073435c7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147442 May 24 10:22:19.872476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 431/882] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:20.205164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:20.206115 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:20.208448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=54 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:20.213173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:20.213656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:20.214577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:20.215745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:20.245191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d4181f72-4c8a-437d-b81e-7c7925cbeefc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043860 May 24 10:22:20.245786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 452/883] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:20.678528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:22:20.678741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:22:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:22:20.889356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:20.889878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:20.896366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:20.896811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:20.897602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:20.898031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:21.029312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:21.036690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bc3f5ea9-f78e-4f3a-98fb-e02477adb45e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151960 May 24 10:22:21.037250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 432/884] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:21.257250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:21.258191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:21.264775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:21.265221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:21.266004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:21.266439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:21.294878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b35b18dd-b0f0-4a1d-baa7-3f8cc5c607da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041935 May 24 10:22:21.295425 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 453/885] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:22.053373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:22.053982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:22.060645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:22.061222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:22.061890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:22.062325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:22.191328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:22.198913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bca66aac-2238-47bc-b954-a777b167f1c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149664 May 24 10:22:22.200018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 433/886] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:22.306618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:22.307454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:22.314800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:22.315243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:22.315995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:22.316425 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:22.345007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-01e61be2-5a60-4ee3-9698-7dfb54301433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042349 May 24 10:22:22.345576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 454/887] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:23.216754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:23.217382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:23.219705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=58 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:23.224394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:23.228017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:23.228017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:23.228017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:23.229913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:23.356089 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:23.356904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:23.358659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:23.363447 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:23.363900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:23.364674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:23.365108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:23.366411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a5f4900e-54cd-444b-9a25-f0ce5f9bbe5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153745 May 24 10:22:23.366990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 434/888] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:23.394334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f8e3d080-2c7c-4fa2-b20a-ebbc3f4780dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042254 May 24 10:22:23.394886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 455/889] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:24.383431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:24.384031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:24.391328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:24.391773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:24.392529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:24.392959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:24.406373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:24.407287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:24.413759 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:24.414268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:24.414967 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:24.415397 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:24.444723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9c9fc064-178e-4332-b442-024d921f3c95 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042763 May 24 10:22:24.445308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 456/890] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:24.526817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:24.534671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-695e2017-57d7-4bbe-a5b8-5b202052109d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155152 May 24 10:22:24.540879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 435/891] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:25.456312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:25.457230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:25.463659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:25.464106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:25.464865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:25.465319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:25.495909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-73f265ff-58c8-43de-968b-add8f57530c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043540 May 24 10:22:25.496476 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 457/892] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:25.551820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:25.552441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:25.558966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:25.559421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:25.560185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:25.560616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:25.690637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:25.698063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-98116604-421b-4d9b-9ab6-fe8b63936473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150370 May 24 10:22:25.698627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 436/893] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:26.507625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:26.508582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:26.515517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:26.515959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:26.516718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:26.517146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:26.701975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cba6fb59-185c-450c-9b81-a56947d0ef06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.198359 May 24 10:22:26.702522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 458/894] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:26.715357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:26.715967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:26.723095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:26.723542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:26.724304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:26.724763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:26.856280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:26.863865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c048e10b-32c7-4905-bbb3-3962ef6897b5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152359 May 24 10:22:26.864429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 437/895] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:27.713629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:27.714537 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:27.721516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:27.721958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:27.722711 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:27.723138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:27.752797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-79b4926d-b341-4103-94cf-2cee41863e74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043156 May 24 10:22:27.753346 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 459/896] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:27.880901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:27.881536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:27.888746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:27.889206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:27.889993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:27.890421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:28.025070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:28.033170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0f8680ce-f1e7-4577-a88b-dc2b7afb4ca4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156062 May 24 10:22:28.033770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 438/897] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:28.765028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:28.766009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:28.772356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:28.772803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:28.773581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:28.774028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:28.803166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-475daa16-05ec-44c1-ba45-78e766e41088 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042676 May 24 10:22:28.803702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 460/898] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:29.050592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:29.051202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:29.057997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:29.058444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:29.059228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:29.060269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:29.198563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:29.207000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a8e0ed6-cb6c-44fb-96f9-22dd3266579e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.160722 May 24 10:22:29.207572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 439/899] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:29.814987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:29.815903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:29.822358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:29.822801 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:29.823550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:29.823975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:29.852712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d2c1ab37-b5cb-42cd-8c08-76ec3395336f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041711 May 24 10:22:29.853367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 461/900] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:30.224259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:30.224861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:30.231315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:30.231927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:30.232742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:30.233176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:30.362636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:30.370195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b0a25654-3307-45d2-8aa4-27194f27b4f9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149859 May 24 10:22:30.371839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 440/901] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:30.864440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:30.865358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:30.871778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:30.872358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:30.873663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:30.874087 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:30.902035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5edaa589-85ec-41c1-a359-ea02ed69f025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041710 May 24 10:22:30.902564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 462/902] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:31.387989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:31.388774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:31.395243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:31.395730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:31.396497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:31.396934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:31.528200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:31.536219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b56166ff-6df8-4df4-9154-d479d43f9669 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152183 May 24 10:22:31.536787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 441/903] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:31.913827 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:31.914666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:31.921588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:31.922045 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:31.922818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:31.923285 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:31.949616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:31.954875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-74823e1b-dee5-4211-a2ca-d5a2ab022394 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045101 May 24 10:22:31.962598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 463/904] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:32.553951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:32.554554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:32.561434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:32.561881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:32.562638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:32.563065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:32.694073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:32.702254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d7b7759d-2015-4f40-a193-33ed0406add2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152388 May 24 10:22:32.702808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 442/905] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:32.972718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:32.973594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:32.980039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:32.980488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:32.981246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:32.981715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:33.011649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ccad8967-8fa6-4709-ab33-47e405a4f9fe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043328 May 24 10:22:33.012211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 464/906] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:33.720375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:33.720981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:33.727518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:33.727968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:33.728726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:33.729161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:33.857903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:33.865303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7acc3ae3-4b95-44b4-a851-90a76c627bb0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149273 May 24 10:22:33.865877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 443/907] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:34.023179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:34.024018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:34.031144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:34.031597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:34.032361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:34.032796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:34.061699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4ff709f1-043a-4fd5-bea6-7f9655156942 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042478 May 24 10:22:34.062248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 465/908] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:34.882115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:34.882637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:34.889470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:34.889917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:34.890670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:34.891099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:35.023599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:35.031582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4acf595-0995-4659-97ec-d46fdc7d62ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153375 May 24 10:22:35.032145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 444/909] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:35.073056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:35.074008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:35.080781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:35.081229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:35.082020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:35.082452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:35.111736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b92b19b-c1de-4b32-9344-ae87c07ecf8e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042555 May 24 10:22:35.112288 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 466/910] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:36.049002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:36.049771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:36.056778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:36.057234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:36.058061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:36.058499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:36.123846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:36.124797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:36.131248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:36.131692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:36.132453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:36.132881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:36.167020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-48532c5a-3429-4b00-8e57-c7142acaaa18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047778 May 24 10:22:36.167843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 467/911] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:36.195405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:36.203574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9942ba3a-9ed4-4ff0-951d-ea88fffc75b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158570 May 24 10:22:36.204145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 445/912] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:37.179484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:37.180409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:37.187575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:37.188024 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:37.188786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:37.189225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:37.218384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1bfc3150-56cd-41e4-9df1-d3d8465e16b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043030 May 24 10:22:37.218938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 468/913] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:37.220912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:37.221607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:37.228241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:37.228697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:37.229568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:37.230012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:37.363872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:37.371346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-99e2aa14-dd3d-44ba-a7f4-fc73d7ffef7b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154400 May 24 10:22:37.372092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 446/914] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:38.229820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:38.230741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:38.237218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:38.237697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:38.238462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:38.238897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:38.267728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9e02847e-98da-4a28-a1d0-7fd298eae583 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041843 May 24 10:22:38.268261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 469/915] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:38.388687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:38.389325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:38.395770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:38.396226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:38.396988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:38.397456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:38.526071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:38.533649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5ad1f4c3-b57d-4bf3-864e-51887e51a653 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148957 May 24 10:22:38.534212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 447/916] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:39.280088 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:39.280915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:39.287360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:39.287841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:39.288602 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:39.289027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:39.322164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-90a4bff5-94ab-4f62-bc77-1adb448c3bb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046671 May 24 10:22:39.322723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 470/917] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:39.551286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:39.551809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:39.558043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:39.558529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:39.559300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:39.560281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:39.693147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:39.701402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-72d8e29f-cdb0-42c3-8420-fcb8f5a8ce1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154239 May 24 10:22:39.701976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 448/918] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:40.334327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:40.335243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:40.342415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:40.342868 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:40.343633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:40.344066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:40.373563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-74d8f042-998d-4fba-ad3c-33754256b5b6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043185 May 24 10:22:40.377058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 471/919] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:40.719437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:40.720075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:40.726637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:40.727097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:40.727865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:40.728305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:40.862417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:40.870056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e0b28ba5-08a5-4c7c-ac7e-9e30da6ca41a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155442 May 24 10:22:40.870632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 449/920] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:41.385072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:41.386022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:41.392663 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:41.393112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:41.393910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:41.394345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:41.423118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6b14e474-5661-45ae-9483-aabd3f71714f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041920 May 24 10:22:41.423593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 472/921] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:41.886665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:41.888025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:41.894928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:41.895377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:41.896138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:41.896776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:42.029920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:42.033356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:42.038016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-50b9e85f-7d7e-46ff-ac1b-a144fcd1c6fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155298 May 24 10:22:42.038789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 450/922] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:42.434813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:42.436402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:42.443139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:42.443613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:42.444392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:42.444858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:42.474865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-96bb153f-4b26-4cff-b40c-b3b516a59d7b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044037 May 24 10:22:42.475422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 473/923] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:43.060486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:43.061099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:43.067622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:43.068077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:43.068841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:43.069293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:43.196554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:43.205109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-95d12b15-2ef9-4be2-b35b-b80ab28d964c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149011 May 24 10:22:43.205738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 451/924] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:43.486296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:43.487211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:43.493739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:43.494192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:43.494995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:43.495428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:43.524004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ec651deb-77ce-4caf-8da5-edf00d5948e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042284 May 24 10:22:43.524590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 474/925] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:44.222432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:44.223114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:44.229641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:44.230090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:44.230855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:44.231287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:44.364695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:44.372120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-db3d8711-dc29-4023-86ad-15ed18dee2f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153576 May 24 10:22:44.372691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 452/926] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:44.535621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:44.536539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:44.543038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:44.543487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:44.544251 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:44.544686 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:44.574990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-40da104f-6c0b-49bb-a4fd-216f515f31cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043271 May 24 10:22:44.575539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 475/927] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:45.390434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:45.391036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:45.397558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:45.398006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:45.398765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:45.399197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:45.527083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:45.534552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-83b85035-d93c-4069-82f6-004758d1d827 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148708 May 24 10:22:45.535121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 453/928] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:45.587121 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:45.588038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:45.594677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:45.595133 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:45.595904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:45.596339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:45.625621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-15fae4cf-68e2-4019-9cc6-7d5442902775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043393 May 24 10:22:45.626171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 476/929] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:46.550727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:46.551335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:46.557804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:46.558249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:46.559007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:46.559433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:46.636367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:46.637319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:46.644276 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:46.644724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:46.645522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:46.645956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:46.676088 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-552f2668-690c-45a4-8e53-eef5ea03b61d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043420 May 24 10:22:46.676641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 477/930] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:46.692101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:46.699564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a24b5567-f460-4a0c-ab0d-feffe068aaba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152983 May 24 10:22:46.700133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 454/931] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:47.691726 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:47.692645 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:47.699192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:47.699636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:47.700389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:47.700818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:47.716849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:47.718030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:47.724503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:47.724953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:47.725753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:47.726190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:47.730329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2ba0b39-1254-44d7-ba72-0889e0cc46ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046608 May 24 10:22:47.730887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 478/932] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:47.856171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:47.864352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4e283c48-7d55-4580-b562-d59d33f6693b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151412 May 24 10:22:47.864934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 455/933] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:48.742942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:48.743857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:48.751386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:48.751828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:48.752582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:48.753011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:48.782013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-677ed38b-6f40-4e68-9b10-4f7dba977dec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043180 May 24 10:22:48.782604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 479/934] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:48.881536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:48.882062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:48.888639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:48.889121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:48.889911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:48.890350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:49.017604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:49.025105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9bf0906e-344c-4f4d-a853-abef444b4493 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147514 May 24 10:22:49.025700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 456/935] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:49.793864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:49.794794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:49.801956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:49.802402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:49.803214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:49.803630 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:49.820225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:49.833107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8c68cf7f-c58d-46c7-afc9-66c7b5cca4cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043307 May 24 10:22:49.836508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 480/936] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:50.042202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:50.042811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:50.049477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:50.049870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:50.050907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:50.051341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:50.182674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:50.190667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8ba4fe34-862d-40f0-9417-0ab8b71afaf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152358 May 24 10:22:50.191233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 457/937] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:50.847039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:50.848104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:50.855243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:50.855681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:50.856430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:50.856855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:50.885867 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4c7b7dcd-694c-4cee-bac8-a4c0dbc6ddb5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043390 May 24 10:22:50.886329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 481/938] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:51.208535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:51.209138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:51.215691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:51.216136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:51.216896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:51.217345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:51.346257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:51.353688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-66fca0cb-fd03-4b1c-9a04-c3244b71edd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149002 May 24 10:22:51.354252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 458/939] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:51.897879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:51.898789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:51.901051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=60 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:51.905847 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:51.906408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:51.907166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:51.907594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:51.936347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e6e09fa4-ff9c-4cae-8022-5ecd92bf04d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042446 May 24 10:22:51.936887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 482/940] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:52.370716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:52.371326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:52.378524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:52.378978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:52.379732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:52.380166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:52.513698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:52.521294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-52cc158a-2483-4857-b25e-582a3ae8bac1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154464 May 24 10:22:52.521860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 459/941] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:52.948689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:52.949642 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:52.956499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:52.956947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:52.957749 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:52.958177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:52.987414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8aa559ee-2680-4a41-b137-414a7aed6eb0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042802 May 24 10:22:52.987961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 483/942] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:53.539125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:53.539729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:53.542062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:53.547454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:53.551934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:53.551934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:53.551934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:53.678578 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:53.686076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c1803013-5908-43d3-8207-696ead297744 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150889 May 24 10:22:53.686649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 460/943] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:53.998594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:53.999509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:54.006585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:54.007027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:54.007779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:54.008213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:54.036050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-98e4b42e-1388-4711-8b1b-48efcacb70bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041399 May 24 10:22:54.036590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 484/944] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:54.704250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:54.704858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:54.711370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:54.711817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:54.712580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:54.713011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:54.850131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:54.857764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a179d0d2-3e9d-4aeb-82bd-1d1cc180fd1e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158268 May 24 10:22:54.858336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 461/945] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:22:55.048692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:55.050087 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:55.057035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:55.057523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:55.058285 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:55.058722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:55.088000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-34c68fec-9a4c-4662-b743-12f81db15d4f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044436 May 24 10:22:55.088553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 485/946] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:55.876434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:55.876969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:55.883419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:55.883870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:55.884629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:55.885063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:56.011625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:56.019820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b922b166-8b87-4cea-b034-9a298e4d277b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148081 May 24 10:22:56.020390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 462/947] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:56.099489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:56.100468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:56.106992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:56.107426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:56.108194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:56.108629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:56.137200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0a16ff1d-4924-41ca-a04d-f722ea6fd577 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041579 May 24 10:22:56.137701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 486/948] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:57.036610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:57.037218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:57.043722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:57.044167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:57.044929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:57.045397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:57.149395 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:57.150317 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:57.157277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:57.157727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:57.162357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:57.162357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:57.175367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:57.183047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-269e63ad-122e-4371-87b7-dcdab39f5e64 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150308 May 24 10:22:57.183628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 463/949] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:57 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:57.190014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1dfbe5a0-fa2e-45c6-b672-3db4ba268322 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044621 May 24 10:22:57.190565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 487/950] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:58.200133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:58.200744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:58.201083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:58.202035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:58.207689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:58.208137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:58.208591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:58.208938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:58.209054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:58.209403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:58.209810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:58.210236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:58.240891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c9fbfa8a-6912-4783-ae62-d50f0582faf3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043624 May 24 10:22:58.241492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 488/951] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:22:58.341217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:58.348676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cdc619e4-168d-4c44-a30e-8ae643505b4b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152593 May 24 10:22:58.349249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 464/952] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:58 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:22:59.253335 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:59.254265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:59.260383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:59.260824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:59.261618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:59.262053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:59.291155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a75432ee-da48-43bd-9420-9a71e314ba16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042745 May 24 10:22:59.297875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 489/953] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:22:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:22:59.366431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:22:59.367048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:22:59.373745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:59.374197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:59.375949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:22:59.376469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:22:59.506334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:22:59.513974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f8230ebf-5b07-4821-a3d4-01ba6fd53af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151520 May 24 10:22:59.514541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 465/954] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:22:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:00.303489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:00.304416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:00.310930 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:00.311381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:00.312307 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:00.312753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:00.342052 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4b4233b8-fb9c-488c-8060-9dca3f5e0b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042859 May 24 10:23:00.342604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 490/955] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:00.531277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:00.531894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:00.538424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:00.538879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:00.539653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:00.540091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:00.879580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:00.887099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5c5b1b14-d736-4345-9ade-6d7d358b0612 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.359829 May 24 10:23:00.887664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 466/956] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:01.354557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:01.355468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:01.362771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:01.363224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:01.363990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:01.364428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:01.392861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-63cfe79b-f920-4d2e-9f4d-6871419d8712 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042258 May 24 10:23:01.393444 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 491/957] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:01.904599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:01.905228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:01.911913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:01.912361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:01.913124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:01.913591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:01.931563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:02.040449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:02.048425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-934427cf-b178-4250-b73e-a354e4c7e05c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147720 May 24 10:23:02.049003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 467/958] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:02.404411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:02.405349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:02.411774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:02.412223 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:02.412978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:02.413446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:02.443212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b84701e4-af84-4dc9-9c94-03837dbd3b64 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042778 May 24 10:23:02.443761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 492/959] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:03.066344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:03.066955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:03.073490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:03.073940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:03.074707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:03.075141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:03.207796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:03.215257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e74cfe53-389d-481e-8d98-393014f89dcc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153189 May 24 10:23:03.215826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 468/960] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:03.455489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:03.456448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:03.462941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:03.463391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:03.464149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:03.464584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:03.493013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23dedfa7-2451-4b44-9ac1-75a50d4292d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042289 May 24 10:23:03.493586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 493/961] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:04.232092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:04.232761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:04.239861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:04.240305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:04.241059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:04.241517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:04.369252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:04.377473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fce2ebec-e1b1-4728-899b-588c24a0925b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149150 May 24 10:23:04.378037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 469/962] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:04.504187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:04.505112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:04.511786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:04.512234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:04.512999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:04.513464 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:04.542876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a8d50e4e-a906-4b0a-af1d-c98400cfeccb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042796 May 24 10:23:04.543431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 494/963] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:05.394208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:05.394774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:05.401360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:05.401819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:05.402662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:05.403113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:05.532380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:05.540087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5b8172f5-47a5-4b93-b3af-51ff7a545bf3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150390 May 24 10:23:05.540661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 470/964] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:05.553842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:05.554771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:05.561278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:05.561751 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:05.562945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:05.563392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:05.592493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-361d2c79-d6dd-4cf8-9b4d-660ef1c48344 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042839 May 24 10:23:05.593039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 495/965] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:06.557739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:06.558576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:06.565332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:06.565776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:06.566538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:06.566966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:06.604289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:06.605497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:06.613052 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:06.613612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:06.614442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:06.615054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:06.648089 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1a20c246-2be8-4da8-b712-6b383f840b27 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047732 May 24 10:23:06.648887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 496/966] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:06.700206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:06.707890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-de701f47-a33c-4542-9d16-1b84f8790c7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154063 May 24 10:23:06.708460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 471/967] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:07.660036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:07.660957 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:07.667436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:07.667885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:07.668645 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:07.669111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:07.673254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:07.698679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d3b6f346-7892-4e2d-8402-8e2ca0bc6516 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043033 May 24 10:23:07.699228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 497/968] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:07.724896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:07.725646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:07.732276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:07.732724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:07.733525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:07.734499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:07.862194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:07.869612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-516e8f64-9c21-4a54-9f46-f67a309a78b3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148492 May 24 10:23:07.870184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 472/969] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:08.710850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:08.711810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:08.719166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:08.719605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:08.720373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:08.720838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:08.749683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f0c13b9f-3db1-4ff9-ae9f-f6dbd20df835 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042829 May 24 10:23:08.750775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 498/970] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:08.887988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:08.888522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:08.895699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:08.896140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:08.896897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:08.897343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:09.027658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:09.035202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8296431e-86f4-4438-8efb-2473584c4dd3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151512 May 24 10:23:09.035800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 473/971] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:23:09.761327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:09.762189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:09.769382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:09.769830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:09.770592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:09.771035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:09.801079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9b4e7f6d-9ba2-4dfe-bc69-858212c3c22c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043559 May 24 10:23:09.801662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 499/972] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:10.052842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:10.053409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:10.059895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:10.060344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:10.061115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:10.061581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:10.192525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:10.200084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5ae8c176-4a8a-42fc-b3a6-5740140c0677 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151434 May 24 10:23:10.200655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 474/973] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:10.812090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:10.813113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:10.820235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:10.820680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:10.821466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:10.821894 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:10.851536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-95d66bbd-e109-4a10-ab61-e12c2a7b16ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043399 May 24 10:23:10.852089 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 500/974] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:11.217707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:11.218313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:11.225695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:11.226142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:11.226897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:11.227328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:11.358180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:11.365900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-88341f7e-2808-4a00-aaed-a25a0ec651f6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152060 May 24 10:23:11.366476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 475/975] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:11.863670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:11.864588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:11.871376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:11.871736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:11.872600 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:11.873035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:11.908014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b592155-81a1-4842-a809-305bae5da0b2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048384 May 24 10:23:11.908619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 501/976] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:12.382756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:12.383352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:12.389816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:12.390259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:12.391009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:12.392108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:12.524990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:12.532450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-16d97e4d-4020-4e28-a064-e377d4b39855 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153506 May 24 10:23:12.533014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 476/977] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:12.920593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:12.921457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:12.928306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:12.928746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:12.929529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:12.929959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:12.958482 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e7376200-7228-45cc-9712-01833499c546 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041925 May 24 10:23:12.959029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 502/978] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:13.549618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:13.550259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:13.557042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:13.557523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:13.558293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:13.558729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:13.687025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:13.694746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b95a7982-aa62-4658-89c8-200aa7031adc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149929 May 24 10:23:13.695234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 477/979] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:13.970168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:13.971042 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:13.977472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:13.977915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:13.978664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:13.979091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:14.007181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2a693d34-b473-4289-9c06-e2ea818610de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041100 May 24 10:23:14.007724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 503/980] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:14.711961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:14.712594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:14.719028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:14.719472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:14.720223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:14.720653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:14.853898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:14.861348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a32cd724-83b7-463d-b92b-cb42021e6a3f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153233 May 24 10:23:14.861914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 478/981] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:15.021295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:15.021295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:15.026681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:15.027860 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:15.028049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:15.028486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:15.058343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e16c111e-d92a-4887-9e71-1e18cfc16f99 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043543 May 24 10:23:15.058890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 504/982] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:15.878210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:15.879416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:15.885955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:15.886397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:15.887156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:15.887584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:16.017922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:16.025732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-26d7a70f-51cc-46fe-8dcd-b0529b0f06e8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151389 May 24 10:23:16.026314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 479/983] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:16.070418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:16.071344 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:16.077808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:16.078264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:16.079075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:16.079522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:16.108148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-33ad3085-d45a-47e7-862e-fec9487427eb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041692 May 24 10:23:16.108698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 505/984] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:17.042805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:17.043413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:17.050774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:17.051484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:17.052238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:17.052665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:17.120084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:17.120997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:17.127517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:17.127971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:17.128722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:17.129149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:17.159888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f47fff3f-5b06-42f8-b242-fe5ea9ed3baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043256 May 24 10:23:17.163368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 506/985] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:17.185249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:17.192850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b23904d4-73eb-4152-b4f7-eb27829f89b6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153931 May 24 10:23:17.193485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 480/986] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:18.170980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:18.171895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:18.179309 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:18.179753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:18.180510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:18.180937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:18.209431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-935e46fb-d274-418d-ae8d-719762f9683f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042413 May 24 10:23:18.209890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:18.210417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 507/987] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:18.210546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:18.217060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:18.217546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:18.218321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:18.218755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:18.352340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:18.359925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-02cc4ec4-5d46-4831-840b-83b5de5bbcec tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153808 May 24 10:23:18.361111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 481/988] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:23:19.220464 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:19.221460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:19.228220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:19.228668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:19.229495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:19.229933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:19.258828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-51eac91d-9d5d-46a3-906e-55b2e5d36312 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042374 May 24 10:23:19.259371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 508/989] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:19.377587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:19.378217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:19.384795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:19.385241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:19.386040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:19.386475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:19.517781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:19.525298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6be4edb9-4f48-459c-8e20-e02da534b8bc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152409 May 24 10:23:19.525873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 482/990] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:20.270002 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:20.270919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:20.277972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:20.278423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:20.279190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:20.279677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:20.308904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7e9db55f-9a3d-4966-abdd-bdc8def2c4f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043354 May 24 10:23:20.312058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 509/991] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:20.542228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:20.542868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:20.549373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:20.549815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:20.550571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:20.551244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:20.684445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:20.692939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c0f6a401-02e8-4bb8-b852-bd979e482259 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154557 May 24 10:23:20.693553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 483/992] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:21.320520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:21.321482 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:21.328026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:21.328508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:21.329498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:21.329950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:21.358652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-43ae7ec9-bfcc-4d1c-99d1-8825962453d7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042036 May 24 10:23:21.359188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 510/993] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:21.710489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:21.711105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:21.717587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:21.718037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:21.718801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:21.719233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:21.723930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:21.847703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:21.855118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9a4fa326-f85c-4e0a-96a9-e22551fdf8d8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148513 May 24 10:23:21.855684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 484/994] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:22.370843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:22.371760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:22.378976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:22.379428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:22.380192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:22.380691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:22.409642 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e3006a5c-ac61-459e-aa62-ed11ff25ab8f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042781 May 24 10:23:22.410190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 511/995] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:22.872727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:22.873240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:22.879852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:22.880296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:22.881052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:22.881609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:23.018563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:23.021761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-736401bf-ef90-4eae-9fa3-3eaa10506d5f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153080 May 24 10:23:23.022400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 485/996] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:23.421628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:23.422494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:23.424777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=60 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:23.429908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:23.430363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:23.431136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:23.431574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:23.461186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-62e06503-a87c-446d-8463-57bd22604261 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043562 May 24 10:23:23.461799 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 512/997] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:24.038073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:24.038671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:24.040973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:24.045851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:24.046296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:24.047052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:24.048007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:24.177815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:24.185226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e6f86078-3a8b-44d9-a0a0-88699d9a3b2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151068 May 24 10:23:24.188932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 486/998] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:24.472772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:24.473725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:24.480354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:24.480801 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:24.481592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:24.482026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:24.506956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:24.511360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bb7ac784-8969-4256-990d-67570d09ff3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043322 May 24 10:23:24.511913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 513/999] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:25.203175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:25.204428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:25.211246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:25.211694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:25.212459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:25.212897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:25.343009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:25.350885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7d2ef2b2-3c74-4d2f-8e9f-20aea0a67271 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151584 May 24 10:23:25.351489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 487/1000] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:25.523360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:25.524299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:25.530778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:25.531220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:25.532173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:25.532550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:25.561050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-07c496b3-321b-482d-ab86-acb459d552f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041928 May 24 10:23:25.561633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 514/1001] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:26.370026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:26.370642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:26.379058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:26.379503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:26.380300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:26.380728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:26.515360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:26.523667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5f1f7601-b523-4a1c-b376-42535879e304 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158679 May 24 10:23:26.524581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 488/1002] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:26.573516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:26.574427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:26.581960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:26.582404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:26.583160 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:26.583587 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:26.613483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-32298464-6cc0-4c51-8eff-593fbbc83ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043865 May 24 10:23:26.614790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 515/1003] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:27.541222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:27.541889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:27.548625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:27.549072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:27.549856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:27.550289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:27.624050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:27.624966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:27.632013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:27.632457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:27.633218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:27.633672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:27.663540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-171d295a-55da-4f3d-b8cc-403e562302a9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043387 May 24 10:23:27.664104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 516/1004] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:27.679970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:27.687472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1f6d1ec5-d092-4d4f-815f-4abedfea9b31 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150066 May 24 10:23:27.688042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 489/1005] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:28.675287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:28.676457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:28.683145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:28.683593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:28.684394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:28.684855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:28.704902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:28.705822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:28.712347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:28.712795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:28.713679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:28.714113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:28.715443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a8aed19f-062c-4174-a6cb-9adf8e6e0e7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044125 May 24 10:23:28.716007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 517/1006] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:28.850073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:28.857582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bc05aae4-6114-4e1f-a2ab-ae5d137a5cb2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.157606 May 24 10:23:28.858148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 490/1007] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:23:29.727244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:29.728271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:29.734883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:29.735325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:29.736784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:29.737390 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:29.765736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-55624202-5baf-4d46-9d22-accb193f4fc6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042380 May 24 10:23:29.766278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 518/1008] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:29.874280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:29.874925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:29.881817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:29.882272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:29.883042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:29.883479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:30.014383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:30.022656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1f22a2ae-0bc7-4525-9ff2-d210ccdf868f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152167 May 24 10:23:30.023284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 491/1009] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:30.777069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:30.778034 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:30.784496 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:30.784937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:30.785723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:30.786150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:30.815775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-018a041a-246a-44de-95e9-501d7b17dbed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042746 May 24 10:23:30.816314 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 519/1010] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:31.040983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:31.041671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:31.048765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:31.049224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:31.050022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:31.050500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:31.182335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:31.190492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-63c1e902-f630-4beb-8485-c791ae465191 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153413 May 24 10:23:31.191051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 492/1011] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:31.827323 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:31.828268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:31.835264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:31.835704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:31.836529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:31.836961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:31.868257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f10979a0-eb75-42ab-a50a-0bc384747bc5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044870 May 24 10:23:31.868808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 520/1012] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:32.208242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:32.208861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:32.215435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:32.215888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:32.216657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:32.217096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:32.346281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:32.353852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ce535d2c-fa67-4182-b779-2ba9a5baef92 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150157 May 24 10:23:32.354406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 493/1013] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:32.879785 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:32.880704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:32.886784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:32.887296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:32.888017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:32.888461 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:32.917332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-195131b3-288e-48b1-b479-17b342c43a2a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042225 May 24 10:23:32.917883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 521/1014] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:33.083518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:23:33.083699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:23:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:23:33.371074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:33.371684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:33.378372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:33.378821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:33.379588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:33.380024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:33.510433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:33.518041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-38a7fae5-03df-43c8-9772-c77ff2da0a41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150788 May 24 10:23:33.518771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 494/1015] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:33.929300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:33.930178 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:33.936753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:33.937203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:33.937997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:33.938432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:33.969588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8d9fc919-5371-48d1-bcb9-898ebb5436cf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044635 May 24 10:23:33.970150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 522/1016] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:34.535252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:34.535789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:34.542331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:34.542817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:34.543605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:34.544037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:34.679788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:34.687474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7f6f1bd5-c820-4c14-b9db-4e0a93ac268e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156255 May 24 10:23:34.688071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 495/1017] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:34.982202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:34.983125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:34.990665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:34.991109 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:34.991869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:34.992334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:35.022942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fb303c73-31c0-42f0-816f-f87d9a8fc0d4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044836 May 24 10:23:35.023493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 523/1018] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:35.706359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:35.706979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:35.713657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:35.714109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:35.714878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:35.715315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:35.842806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:35.850925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-146d3928-36da-4ab1-9094-94aaec4bbdf1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149384 May 24 10:23:35.851497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 496/1019] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:36.034858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:36.035601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:36.042058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:36.042509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:36.043269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:36.043704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:36.073105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-04ad72d8-1428-4d02-9599-214491781e61 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042446 May 24 10:23:36.073679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 524/1020] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:36.867706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:36.868291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:36.875050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:36.875497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:36.876252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:36.876678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:37.012323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:37.020539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a38f100d-224c-477b-b97e-d0405a6d642e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.156731 May 24 10:23:37.021103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 497/1021] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:37.085075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:37.086021 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:37.092442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:37.092888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:37.093679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:37.094116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:37.122563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4592fbe6-2a8e-4b79-9262-6988c450f505 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042019 May 24 10:23:37.123110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 525/1022] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:38.038102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:38.038739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:38.045283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:38.045751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:38.046535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:38.047009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:38.134792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:38.135833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:38.143545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:38.144047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:38.145023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:38.145661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:38.179308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-55fd35ff-2c98-4756-9e74-f7c74a479f10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048524 May 24 10:23:38.179878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 526/1023] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:38.260260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:38.268027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-67a53488-720d-444a-9b3a-0be7fedbbabb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.234061 May 24 10:23:38.268610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 498/1024] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:39.191482 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:39.192446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:39.198986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:39.199441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:39.200209 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:39.200649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:39.230781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-522ad03e-9eca-4b7a-86b3-bceb7d9ee726 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043812 May 24 10:23:39.231329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 527/1025] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:39.285575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:39.286114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:39.292256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:39.292703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:39.293501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:39.293936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:39.423587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:39.431311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-751a1e15-dfd7-4173-bdbb-aea1122c8d19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150325 May 24 10:23:39.431873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 499/1026] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:40.243027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:40.243934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:40.250627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:40.252274 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:40.253097 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:40.253578 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:40.282264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c385a8f9-4805-49c9-85de-a8452ac41c80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043033 May 24 10:23:40.282817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 528/1027] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:40.448281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:40.448899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:40.456413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:40.456898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:40.457717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:40.458155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:40.591463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:40.594858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:40.599881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4d341d93-db07-48a4-a483-ddc1f9fa6309 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155549 May 24 10:23:40.600444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 500/1028] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:41.294414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:41.295334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:41.302676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:41.303125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:41.303886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:41.304317 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:41.333647 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1574b3da-5619-4bf0-a9a5-52e0c0bf2d7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043195 May 24 10:23:41.334190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 529/1029] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:41.618615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:41.619219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:41.626371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:41.626817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:41.628172 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:41.628607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:41.760178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:41.768373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4d32791d-a65e-4655-8a4e-27bbb2dd24be tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152941 May 24 10:23:41.769136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 501/1030] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:42.344688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:42.345701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:42.352236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:42.352694 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:42.353483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:42.353923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:42.370146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:42.383131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2cada990-87d9-465a-9d93-a4a7d6a4b024 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042523 May 24 10:23:42.388124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 530/1031] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:42.786201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:42.786805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:42.793281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:42.793733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:42.794487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:42.794915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:42.936292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:42.943853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b5c13452-e914-4780-84e8-84474cf08518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.162287 May 24 10:23:42.944417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 502/1032] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:43.399854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:43.399854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:43.406147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:43.406596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:43.407354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:43.407790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:43.438143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fe5bb253-b4cf-4e0c-950d-fc6af384a203 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043813 May 24 10:23:43.440053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 531/1033] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:43.962080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:43.962694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:43.969184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:43.969667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:43.970428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:43.970890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:44.099918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:44.107726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a96683e4-03f9-4c0b-aec0-312128fba424 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149541 May 24 10:23:44.108294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 503/1034] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:44.449817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:44.450763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:44.457489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:44.457931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:44.458683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:44.459107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:44.488114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fbc794a6-dac8-4eb4-b221-e5a22b237afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042272 May 24 10:23:44.488658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 532/1035] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:45.124681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:45.125303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:45.131988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:45.132436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:45.133196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:45.133654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:45.262024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:45.270401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cbc2adc1-779f-4aeb-87dd-8bcf2c2e55fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149513 May 24 10:23:45.271005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 504/1036] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:45.499797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:45.500719 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:45.507893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:45.508404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:45.509198 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:45.509659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:45.538756 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b6c1350-9bca-4112-9d26-48ad07871c58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042969 May 24 10:23:45.539291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 533/1037] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:46.288478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:46.289213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:46.295691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:46.296139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:46.296905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:46.297384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:46.428301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:46.435789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef35c8a8-2039-479e-980a-c23f4d0cc296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152434 May 24 10:23:46.436390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 505/1038] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:46.550362 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:46.551286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:46.558132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:46.558694 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:46.559423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:46.559855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:46.588964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bdb2a1cd-f795-46db-ba07-a0ddc29139e4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042688 May 24 10:23:46.589539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 534/1039] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:47.453118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:47.453771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:47.460988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:47.461484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:47.462252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:47.462687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:47.600154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:47.601242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:47.607633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:47.608079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:47.608842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:47.609285 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:47.637626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:47.639422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dd1ca8b3-f7dc-4268-a059-6b33eb556a3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043335 May 24 10:23:47.639997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 535/1040] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:47.646249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a2fa452f-592d-42f4-a99d-ef358266681b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.197018 May 24 10:23:47.646819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 506/1041] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:48.652789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:48.653884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:48.660778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:48.661367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:48.662157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:48.662861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:48.663451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:48.664258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:48.670849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:48.671293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:48.672061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:48.672489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:48.694833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-645b297b-bf62-446e-a251-e249824aaad6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047191 May 24 10:23:48.698620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 536/1042] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:48.806793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:48.814468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-85b530f4-d6b6-4c0d-8f61-78a49f30d473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154976 May 24 10:23:48.815833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 507/1043] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:49.706936 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:49.707874 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:49.714332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:49.721509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:49.721509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:49.721509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:49.744489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ba637998-3fde-4c92-b6d5-72c9b10ad011 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042003 May 24 10:23:49.745022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 537/1044] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:49.832302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:49.833607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:49.840175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:49.840625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:49.841420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:49.841857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:49.971876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:49.979365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4078a027-6903-46c5-9889-5f7c93de1f11 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151091 May 24 10:23:49.979945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 508/1045] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:50.756006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:50.756928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:50.763670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:50.764113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:50.764878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:50.765422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:50.795284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a48faaf3-d2a9-444a-ba81-1704536cdf51 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043275 May 24 10:23:50.795825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 538/1046] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:50.996332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:50.996927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:51.003584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:51.004032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:51.004782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:51.005209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:51.131501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:51.138864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2ca5f440-225c-497e-beff-fbf20c9ee1fc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.146353 May 24 10:23:51.139433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 509/1047] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:51.808268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:51.809197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:51.815795 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:51.817111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:51.818115 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:51.818560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:51.846864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7a8f8adc-49fd-41b1-ad7a-beda2e214e3d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042583 May 24 10:23:51.847493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 539/1048] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:52.156037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:52.156597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:52.163706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:52.164152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:52.164918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:52.165789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:52.296877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:52.305716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-60bd114b-0ec8-4570-aa9f-eb09d9377659 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153629 May 24 10:23:52.306294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 510/1049] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:52.858738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:52.859649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:52.866300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:52.866746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:52.867515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:52.867948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:52.896982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-032aa76b-ade0-4f65-af8a-022bd8b77ea7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042285 May 24 10:23:52.897563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 540/1050] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:53.323198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:53.323809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:53.330508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:53.330956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:53.331760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:53.332192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:53.460786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:53.468505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-175deb50-023b-458d-98cb-556bb95e9c61 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149147 May 24 10:23:53.469108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 511/1051] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:53.973438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:53.974359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:53.980456 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:53.980909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:53.981705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:53.982141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:54.010011 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0b7609b7-7a39-4455-836f-110e2432798b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.105503 May 24 10:23:54.010552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 541/1052] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:54.485705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:54.486305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:54.488383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:54.493256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:54.499001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:54.499001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:54.499001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:54.627950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:54.636192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5adb2542-1793-4b84-b991-55ac44ea1aeb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155067 May 24 10:23:54.636760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 512/1053] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:55.021074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:55.027365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:55.027365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=60 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:55.029081 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:55.029564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:55.030331 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:55.030759 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:55.060905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f91e622b-a2f8-4269-8eb9-a67844cda0e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043852 May 24 10:23:55.061485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 542/1054] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:55.653397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:55.654003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:55.661744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:55.662191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:55.662951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:55.663393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:55.790762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:55.798209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ca4b98e8-6791-4a46-be23-20d456c07716 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148755 May 24 10:23:55.798776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 513/1055] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:56.072782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:56.073710 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:56.079988 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:56.080419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:56.081164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:56.081650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:56.111222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1a7194af-b4a5-4e88-8f6a-bbed2a225137 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042691 May 24 10:23:56.111770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 543/1056] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:56.816950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:56.817805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:56.825945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:56.826392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:56.827146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:56.827575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:56.963734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:56.971250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-27f5d604-76b4-4278-93f6-c81fb6542df0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158987 May 24 10:23:56.971821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 514/1057] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:57.123381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:57.124319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:57.130841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:57.131284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:57.132039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:57.132468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:57.162230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-083e8454-d868-44c8-bf19-5d6832fd6916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042673 May 24 10:23:57.162696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 544/1058] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:57.372876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:23:57.373088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:23:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:23:57.987976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:57.988727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:57.995281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:57.995757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:57.996549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:57.996984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:58.123703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:58.131697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96f793ee-add0-4a56-ad2e-a1d840255da5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147710 May 24 10:23:58.132258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 515/1059] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:57 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:23:58.173776 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:58.174716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:58.181830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:58.182273 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:58.183025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:58.183456 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:58.225584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f93c378d-1f25-49e6-9632-9800ea05beeb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055627 May 24 10:23:58.226116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 545/1060] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:23:59.148953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:59.149617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:59.156058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:59.156525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:59.157308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:59.157727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:59.237111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:23:59.238790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:23:59.245252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:59.245731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:59.246495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:23:59.246949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:23:59.277229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-29d151b5-5ae2-4735-b48d-9bc7c93165fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044065 May 24 10:23:59.277881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 546/1061] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:23:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:23:59.291162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:23:59.298836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7d6790b7-54d4-4067-9307-99127deec882 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153756 May 24 10:23:59.299403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 516/1062] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:23:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:00.288620 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:00.289562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:00.296012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:00.296452 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:00.297205 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:00.297655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:00.301605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:00.315729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:00.316882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:00.323709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:00.324159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:00.324936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:00.325396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:00.329129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-964518c1-090f-4200-9611-7459b957faac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044551 May 24 10:24:00.329739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 547/1063] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:00.344348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:00.459796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:00.467088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9361b9ea-2bea-4c85-87f5-29ad3c1ee086 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155152 May 24 10:24:00.467652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 517/1064] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:01.342228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:01.343152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:01.350075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:01.350525 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:01.351281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:01.351714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:01.381344 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5a8a9eeb-04e4-4e9a-a130-72c4d7838101 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043057 May 24 10:24:01.381895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 548/1065] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:01.485030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:01.485670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:01.493523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:01.493967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:01.494804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:01.495231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:01.622830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:01.630566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-53f9f333-a8e9-4fea-8503-b08b97196a20 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149378 May 24 10:24:01.631133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 518/1066] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:02.392606 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:02.393546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:02.399565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:02.400015 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:02.400773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:02.401238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:02.430510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5d83c5f1-32a2-427d-a3c2-f9d5aae990b7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041955 May 24 10:24:02.430993 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 549/1067] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:02.647628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:02.648231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:02.654707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:02.655152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:02.655950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:02.656325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:02.786923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:02.794442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fc9aeb0b-b635-4f58-9c4c-6114357db308 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150795 May 24 10:24:02.795034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 519/1068] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:03.442746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:03.443661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:03.450138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:03.450586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:03.451343 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:03.451779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:03.481321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d0e30ca5-5aa8-4019-a2eb-db80fbf68474 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043315 May 24 10:24:03.481869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 550/1069] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:03.812339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:03.812863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:03.819339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:03.819786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:03.820545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:03.820978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:03.948846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:03.956342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4dbaae4-c7a4-48b9-a32e-55dd4b4490c4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147980 May 24 10:24:03.957444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 520/1070] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:04.492535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:04.493484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:04.499905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:04.500353 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:04.501117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:04.501577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:04.530207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4658dbf4-2fda-4fff-80b1-96fabb39e7f6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042144 May 24 10:24:04.530747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 551/1071] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:04.974371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:04.974964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:04.981486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:04.981933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:04.982699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:04.983131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:05.115874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:05.123544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6cbf05ae-38eb-4b45-abfb-89824acccded tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153511 May 24 10:24:05.124107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 521/1072] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:05.541897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:05.542825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:05.549258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:05.549733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:05.551202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:05.551725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:05.580460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7ae880aa-1e6b-491f-b66c-0a7c68f2696c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042551 May 24 10:24:05.581009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 552/1073] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:06.141311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:06.141931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:06.149226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:06.149706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:06.150475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:06.150905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:06.283283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:06.290931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-30f9b46f-811a-4a4d-bc5d-874249e003d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153474 May 24 10:24:06.291499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 522/1074] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:06.592477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:06.593440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:06.600489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:06.600943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:06.601773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:06.602214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:06.633702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-74d8e60c-f258-4aec-9fa4-7206f9506ed3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045248 May 24 10:24:06.634299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 553/1075] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:07.307810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:07.308497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:07.315187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:07.315639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:07.316480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:07.316919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:07.444728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:07.452135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef39402d-b6a9-4e6f-abba-0f2bfc281155 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148282 May 24 10:24:07.452693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 523/1076] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:07.646438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:07.647385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:07.653774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:07.654216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:07.654982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:07.655400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:07.684280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f65561b8-42f3-4600-a50c-ed865c81ff84 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042457 May 24 10:24:07.684821 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 554/1077] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:08.469459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:08.470069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:08.477705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:08.478151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:08.478913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:08.479340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:08.615999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:08.623439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-18d409b2-d95b-4cc8-91ba-556c44172616 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.157857 May 24 10:24:08.624005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 524/1078] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:08.695584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:08.696512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:08.703010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:08.703455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:08.704221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:08.704661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:08.733658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7454fe78-27e2-4784-b55c-57a46aec16a5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042276 May 24 10:24:08.734140 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 555/1079] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:09.640047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:09.640660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:09.647284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:09.647733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:09.648493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:09.648936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:09.744962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:09.745923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:09.752364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:09.752816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:09.754269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:09.754740 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:09.777686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:09.783898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d46b1af5-2ea0-4cea-8fee-5ac04a0b2a1a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042772 May 24 10:24:09.784465 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 556/1080] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:24:09.785526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f6c618c5-a4f0-400a-851d-378d85759a55 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149336 May 24 10:24:09.786275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 525/1081] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:10.795576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:10.796505 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:10.802515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:10.803132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:10.803829 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:10.804288 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:10.805064 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:10.805531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:10.809646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:10.810100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:10.810932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:10.811370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:10.836386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f5af99cf-d9d3-4992-a0ea-5dd3f6469955 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044818 May 24 10:24:10.836933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 557/1082] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:10.943167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:10.950553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f1627fb5-4ada-4fcd-9a0b-60e0c5201e57 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152329 May 24 10:24:10.951118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 526/1083] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:11.847499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:11.848423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:11.855074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:11.855519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:11.856312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:11.856755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:11.886296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d625396d-a8b3-412d-9b43-546b7009d4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042745 May 24 10:24:11.886771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 558/1084] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:11.968042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:11.968646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:11.975286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:11.975734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:11.976492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:11.976927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:12.107080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:12.114776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2b5f9e2b-9cec-401a-a3a9-28e5d3896b19 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150617 May 24 10:24:12.115348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 527/1085] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:12.897146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:12.898094 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:12.904540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:12.904990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:12.905787 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:12.906228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:12.936059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-857607ba-cd0c-42a2-a770-bcb4d134741f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042943 May 24 10:24:12.936608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 559/1086] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:13.131741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:13.132350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:13.138847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:13.139307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:13.140071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:13.140508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:13.269799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:13.277464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5cca3df2-9dc8-4120-a739-5e1b02d8aa3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149698 May 24 10:24:13.278040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 528/1087] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:13.948213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:13.949142 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:13.955615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:13.956067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:13.957575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:13.958060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:13.986678 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0ecdfa9f-c2e5-4e69-af10-0900cd414298 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042504 May 24 10:24:13.987220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 560/1088] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:14.295550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:14.296117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:14.303484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:14.303927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:14.304684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:14.305144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:14.436269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:14.444542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3bab6c43-91f2-434f-832a-21539ad60a0d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152962 May 24 10:24:14.445106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 529/1089] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:14.998198 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:14.999117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:15.005723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:15.006173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:15.006932 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:15.007398 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:15.037915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f011f6fb-bf81-4774-9f41-098b6eebd75a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043632 May 24 10:24:15.038462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 561/1090] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:15.461676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:15.462286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:15.468653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:15.469099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:15.469895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:15.470330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:15.599207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:15.606622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a1107ddc-489f-431e-84dd-ac50986a8e68 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148891 May 24 10:24:15.607182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 530/1091] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:16.049427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:16.050889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:16.057745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:16.058193 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:16.058952 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:16.059386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:16.088565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-20677070-d719-48be-ab43-370c66d1454d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043087 May 24 10:24:16.089104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 562/1092] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:16.624120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:16.624754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:16.631370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:16.631816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:16.632632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:16.633062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:16.764070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:16.772229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8aa50337-33db-4171-b18e-931ffd2c5c05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151995 May 24 10:24:16.772799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 531/1093] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:17.100023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:17.100972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:17.107736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:17.108183 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:17.108973 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:17.109441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:17.134523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:17.138857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-84bdd497-69a6-4f37-b7e9-2e8290c99c05 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042852 May 24 10:24:17.139412 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 563/1094] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:17.789403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:17.790011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:17.796475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:17.796923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:17.797711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:17.798146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:17.926400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:17.934263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aba498ef-a59c-4589-9d4f-991d05b9d4f4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148798 May 24 10:24:17.934864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 532/1095] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:18.151664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:18.152535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:18.159059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:18.160254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:18.161103 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:18.161571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:18.190878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d7a83686-6621-4d72-847a-700b8b46921b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043489 May 24 10:24:18.191694 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 564/1096] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:19.004691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:19.005355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:19.011801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:19.012244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:19.012994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:19.013463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:19.140362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:19.148445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e3d9c5f3-9180-4bbd-b8ac-3662aeb5da27 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.148159 May 24 10:24:19.149013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 533/1097] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:19.202075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:19.203016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:19.210272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:19.210729 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:19.211495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:19.211932 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:19.240100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-68b24471-861e-489d-b9cc-68e23f98010b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041967 May 24 10:24:19.240635 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 565/1098] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:20.165744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:20.166360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:20.172871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:20.173351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:20.174117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:20.174551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:20.179425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:20.251540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:20.252571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:20.260337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:20.260861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:20.261910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:20.262513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:20.294213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-84975902-526f-4486-8b64-c630d1f95327 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046516 May 24 10:24:20.294778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 566/1099] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:24:20.311865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:20.320198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-be20e682-e2c9-450a-bce3-e1271fb30ba6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.158353 May 24 10:24:20.321379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 534/1100] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:21.306624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:21.307536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:21.313963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:21.314409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:21.315159 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:21.315588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:21.337293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:21.337913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:21.344326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:21.344529 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aa271d09-d471-451c-87c0-8f717229c2cc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042121 May 24 10:24:21.344781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:21.345086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 567/1101] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:21.345570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:21.346005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:21.472598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:21.480715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d2d0605d-b48a-4b64-b7a5-fc6ff649b534 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147312 May 24 10:24:21.481299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 535/1102] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:22.356987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:22.357929 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:22.364920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:22.365394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:22.366154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:22.366582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:22.395269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0f76240b-d9d4-4e22-b94b-637e190e80c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042179 May 24 10:24:22.395812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 568/1103] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:22.498500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:22.499114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:22.506449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:22.506898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:22.507661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:22.508096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:22.639769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:22.647244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cb241a10-b0bc-4ae0-97bc-367cdc219db4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152571 May 24 10:24:22.647813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 536/1104] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:23.407005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:23.407966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:23.415442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:23.415796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:23.416555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:23.416985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:23.447462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fa88aa5f-af2f-4465-bef9-0f27faaf480f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044551 May 24 10:24:23.448012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 569/1105] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:23.664357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:23.664970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:23.671605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:23.671985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:23.672791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:23.673229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:23.803676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:23.811113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8f3d65bf-6fbe-4be8-92d1-eb07365881f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151056 May 24 10:24:23.811686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 537/1106] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:24.459657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:24.460615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:24.467101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:24.467548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:24.468310 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:24.468750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:24.497977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-735a0595-e34b-4327-ba98-e3a2d3094b2e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042916 May 24 10:24:24.498525 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 570/1107] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:24.827978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:24.828505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:24.830796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:24.835646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:24.836088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:24.836842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:24.837288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:24.969392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:24.977711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6597c458-0b0f-4c83-bd5c-ace4eba7f83f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153689 May 24 10:24:24.978286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 538/1108] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:25.510118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:25.511025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:25.517488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:25.517931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:25.518683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:25.519111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:25.548748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-153f6733-dd22-4a23-bfb9-b5a21cee0361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042629 May 24 10:24:25.549333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 571/1109] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:25.995163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:25.995773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:26.002250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:26.002690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:26.003443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:26.003870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:26.133249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:26.140859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-40603417-8f2b-492b-992c-edb6723793b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149554 May 24 10:24:26.141444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 539/1110] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:26.560124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:26.561370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:26.563545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=60 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:26.568989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:26.569475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:26.570249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:26.570687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:26.600531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f1fd0ea4-6a09-4e88-bb61-5a690e37796f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044340 May 24 10:24:26.601077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 572/1111] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:27.157298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:27.157910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:27.164555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:27.165360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:27.165797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:27.166222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:27.293407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:27.300861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb88d86d-f694-4171-9ba1-c4d80ddfeae4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.147466 May 24 10:24:27.302672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 540/1112] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:27.613373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:27.615368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:27.624061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:27.624618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:27.625526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:27.626119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:27.655727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-87894ac5-75e8-4a58-ace0-90bd983a7964 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047214 May 24 10:24:27.656279 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 573/1113] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:28.319674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:28.320380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:28.326939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:28.327385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:28.328148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:28.328580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:28.462963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:28.470485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6260dfc0-b4e5-4caa-818d-df536731d2a8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155194 May 24 10:24:28.471061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 541/1114] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:28.667518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:28.668431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:28.674944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:28.675383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:28.676136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:28.677179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:28.705760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e9433805-64f7-407e-8d5f-05fee1c55f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042167 May 24 10:24:28.706298 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 574/1115] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:29.487719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:29.488329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:29.495750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:29.496197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:29.496963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:29.497424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:29.628773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:29.636862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-80b3b0f7-4bed-46ff-af2b-301554606939 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.152972 May 24 10:24:29.637486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 542/1116] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:29.717551 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:29.718471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:29.724961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:29.725453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:29.726225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:29.726658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:29.755167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dea3f1bf-2140-4c9d-b0ef-e10ff736b1a1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040976 May 24 10:24:29.755542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 575/1117] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:30.654847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:30.655564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:30.661993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:30.662458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:30.663218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:30.663651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:30.766716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:30.767637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:30.774629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:30.775079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:30.775847 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:30.776284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:30.796642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:30.804468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3fbe119b-4e02-41f3-990a-bb700978f623 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.153519 May 24 10:24:30.805053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 543/1118] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:30.806168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5b3f47cc-4aef-42a7-9603-792f6b972873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043424 May 24 10:24:30.806720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 576/1119] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:24:31.818652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:31.819788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:31.822376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:31.822996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:31.826431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:31.826884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:31.827664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:31.828160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:31.829412 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:31.829865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:31.830631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:31.831077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:31.859179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef738500-8a92-465b-8bbe-c94ce2e897db tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045469 May 24 10:24:31.859755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 544/1120] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:32.010029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:32.017406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1d464774-4e36-439f-bf20-41755831d127 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.198956 May 24 10:24:32.017971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 577/1121] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:32.871698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:32.872706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:32.880313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:32.880755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:32.881566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:32.881997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:32.910749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aab972fa-3491-4a3e-a230-40ef8969d76a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043749 May 24 10:24:32.911298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 545/1122] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:33.035292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:33.035910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:33.043270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:33.043836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:33.044607 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:33.045053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:33.174067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:33.182180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5a85e1a0-6595-43e9-af0e-55213c3c99ff tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151130 May 24 10:24:33.182738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 578/1123] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:33.922634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:33.923553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:33.931064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:33.931547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:33.932309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:33.932740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:33.962119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb6d005e-010f-4fe6-9722-f7ab1125f5ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043459 May 24 10:24:33.962668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 546/1124] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:34.199596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:34.200204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:34.206717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:34.207161 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:34.207915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:34.208420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:34.342630 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:34.350930 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-761e712f-6be0-411a-82cf-959d3b027cde tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.155437 May 24 10:24:34.351481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 579/1125] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:34.973088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:34.974079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:34.980728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:34.981171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:34.981971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:34.982403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:35.011469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0f615914-15c5-4f48-a756-41baeaf72a6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042371 May 24 10:24:35.012034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 547/1126] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:35.368828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:35.369415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:35.382067 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:35.382510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:35.383275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:35.383703 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:35.408558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:35.527286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:35.534782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-35e71689-3036-4de2-aea0-71efd8a8fe0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.170707 May 24 10:24:35.535349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 580/1127] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:36.023683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:36.024629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:36.031187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:36.031640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:36.032411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:36.032848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:36.062213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4437e92c-89b5-42f5-bb5d-9391a43d02c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043189 May 24 10:24:36.062769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 548/1128] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:36.551978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:36.553557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:36.560025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:36.560472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:36.561222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:36.561705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:36.694036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:36.702432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d215edb1-eaaa-4e50-a723-e292b3e5c0b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154358 May 24 10:24:36.702982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 581/1129] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:37.073888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:37.074809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:37.081204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:37.081682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:37.082443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:37.082875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:37.112125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0fc50964-d976-4f21-a995-8f1bcc518164 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042135 May 24 10:24:37.112682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 549/1130] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:37.719779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:37.720437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:37.727625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:37.728066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:37.729090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:37.729543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:37.867772 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:37.875453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-01196da2-f5a8-4be7-85ee-222efdabf001 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.159636 May 24 10:24:37.876008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 582/1131] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:38.124720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:38.125601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:38.132161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:38.132603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:38.133414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:38.133831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:38.159019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:38.164129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1ba96216-6085-40a7-80e5-8c23145c3c7d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043500 May 24 10:24:38.164697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 550/1132] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:38.891638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:38.892251 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:38.899674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:38.900128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:38.900899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:38.901365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:39.034290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:39.042853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8bcc7f2f-fee4-41b9-ab25-9e6b82385367 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154975 May 24 10:24:39.043418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 583/1133] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:39.175660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:39.176581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:39.183525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:39.183974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:39.184737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:39.185169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:39.217163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1630bc6c-8720-42d1-9e80-68ce093047c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045318 May 24 10:24:39.217936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 551/1134] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:40.059667 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:40.060261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:40.066822 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:40.067269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:40.068077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:40.068518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:40.202151 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:40.209976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c14e6696-e148-452d-b58f-6c95f9673625 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.154201 May 24 10:24:40.210552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 584/1135] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:40.228687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:40.229684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:40.236050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:40.236522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:40.237316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:40.237746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:40.267692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f3268573-aba0-4ce6-8fa6-8108c4ea4367 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043669 May 24 10:24:40.268255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 552/1136] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:41.227964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:41.228573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:41.235040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:41.235485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:41.236241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:41.236671 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:41.280001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:41.280938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:41.288177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:41.288620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:41.289413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:41.289850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:41.319688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-542328a5-c8e6-4932-864e-c15149ca9318 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043632 May 24 10:24:41.320264 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 553/1137] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:24:41.365932 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:41.373423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cb9ba577-76de-4397-908f-7fef89cc7ff9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.149434 May 24 10:24:41.373995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 585/1138] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:42.331233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:42.332175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:42.339608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:42.340173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:42.340966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:42.341553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:42.371520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cbc8dc86-f7dd-4413-a2fb-b28a8b3b4020 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044293 May 24 10:24:42.373597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 554/1139] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:42.390651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:42.391291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:42.398434 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:42.398884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:42.399708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:42.400147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:42.539579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:42.547128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5aab2f8b-5dbc-4e6e-baae-85c8477c8ae1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.160278 May 24 10:24:42.547696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 586/1140] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:43.384403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:43.385341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:43.392003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:43.396230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:43.396230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:43.396230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:43.423615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eced413a-0d6b-46f7-8a33-12674c5baf1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044318 May 24 10:24:43.424102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 555/1141] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:43.566049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:43.566685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:43.573530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:43.574007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:43.574704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:43.575134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:43.705782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:43.713213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f7250076-fb8d-45ed-95a8-c56be066eba8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151004 May 24 10:24:43.713805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 587/1142] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:44.436093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:44.437013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:44.444093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:44.444542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:44.445328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:44.445765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:44.474942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-74d25801-53ef-4c72-a9ba-ba5e248538d8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042790 May 24 10:24:44.475519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 556/1143] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:44.730215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:44.730758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:44.737152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:44.737624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:44.738377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:44.738803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:44.868865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:44.876540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f3fbaccb-0c9f-46db-aac2-c622dc1dddf6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.150225 May 24 10:24:44.877094 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 588/1144] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:45.486464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:45.487388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:45.494035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:45.494485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:45.495292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:45.495727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:45.525134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2d5d8c1e-733d-4528-8a1d-74f04d7693a0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042620 May 24 10:24:45.525772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 557/1145] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:24:45.894376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:45.894925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:45.901983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:45.902472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:45.903235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:45.903672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:46.034322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:46.042267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2d34bbdc-c60d-4f04-bb57-0f0e4c2b21dc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.151834 May 24 10:24:46.042828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 589/1146] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:46.538534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:46.539572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:46.545990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:46.546436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:46.547220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:46.547654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:46.580009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8628cd02-6840-490d-9317-4d4f4fd70de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047429 May 24 10:24:46.584067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 558/1147] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:46.953051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:46.953705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:46.961169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:46.961656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:46.962434 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:46.962872 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:46.990251 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Going to try to terminate instance {{(pid=118233) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:24:47.017284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-7c638d76-965f-4f09-8023-cb1296d5f605 req-d8df4f1e-05c5-4723-80a9-e38fcd8dd852 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "426e8866-95a1-42e7-9e88-29f93c5e7d7b", "tag": "0ed9cac0-6315-452a-9289-41b7a7958b3c"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:24:47.029869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7c638d76-965f-4f09-8023-cb1296d5f605 req-d8df4f1e-05c5-4723-80a9-e38fcd8dd852 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:47.030320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7c638d76-965f-4f09-8023-cb1296d5f605 req-d8df4f1e-05c5-4723-80a9-e38fcd8dd852 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:47.042750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-7c638d76-965f-4f09-8023-cb1296d5f605 req-d8df4f1e-05c5-4723-80a9-e38fcd8dd852 service nova] Creating event network-changed:0ed9cac0-6315-452a-9289-41b7a7958b3c for instance 426e8866-95a1-42e7-9e88-29f93c5e7d7b on hv-cinder-84311 May 24 10:24:47.050391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-7c638d76-965f-4f09-8023-cb1296d5f605 req-d8df4f1e-05c5-4723-80a9-e38fcd8dd852 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099129 May 24 10:24:47.051201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 559/1148] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:24:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:47.326088 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aa281374-145f-4d2a-9719-a379c6be8928 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "DELETE /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 204 len: 0 microversion: 2.1 time: 0.376757 May 24 10:24:47.326766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 590/1149] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:24:46 2022] DELETE /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:24:47.336432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:47.337137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:47.344074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:47.344594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:47.348727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:47.348727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:47.571650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:47.579487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96abf8ec-f0f7-4143-946f-0ae330bba0e4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1565 microversion: 2.1 time: 0.247972 May 24 10:24:47.580146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 560/1150] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1565 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:47.592526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:47.594839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:47.602666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:47.603360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:47.604224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:47.604661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:47.641498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-58b65431-6f37-4b06-9523-a72e2d75688c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053414 May 24 10:24:47.642202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 591/1151] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:48.597212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:48.597848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:48.605001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:48.605482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:48.606242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:48.606677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:48.656261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:48.657968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:48.670779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:48.671637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:48.673061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:48.673851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:48.723408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-01a8141b-cbbb-435b-8e02-e6c331ec42ac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073746 May 24 10:24:48.724881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 592/1152] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:24:48.767649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:48.776122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-746e99a5-cbbf-4cf7-8888-aa327c67efdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.182738 May 24 10:24:48.779618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 561/1153] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:49.735232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:49.736145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:49.743106 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:49.743547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:49.744301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:49.744729 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:49.773812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c550a5e7-1d11-4468-8a9e-58dc1f7bbaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042666 May 24 10:24:49.774358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 593/1154] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:49.793464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:49.794074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:49.800459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:49.800902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:49.801690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:49.802166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:49.934654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:49.942069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d29387ef-ec94-4ffa-8ea2-962c17854b4c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152570 May 24 10:24:49.942670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 562/1155] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:50.785589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:50.786808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:50.793449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:50.793899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:50.794657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:50.795090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:50.824614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-49f53eab-c719-4316-a161-8d982c099a22 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043080 May 24 10:24:50.825162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 594/1156] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:50.959073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:50.959678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:50.966436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:50.966885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:50.967640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:50.968073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:51.095052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:51.103345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-daa7f7a8-83c1-4507-852f-9c7bbfec1187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148417 May 24 10:24:51.103925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 563/1157] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:51.837153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:51.838102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:51.844615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:51.845063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:51.845846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:51.846281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:51.864278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:51.876700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5d226d89-20aa-4575-bab8-fb9c827062be tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044195 May 24 10:24:51.886857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 595/1158] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:52.120749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:52.121328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:52.128068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:52.128508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:52.129285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:52.129721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:52.262828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:52.270288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f78cc6d0-42dd-4179-b1c7-655558557aa4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153517 May 24 10:24:52.270952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 564/1159] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:52.897361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:52.898313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:52.905084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:52.905551 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:52.906309 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:52.906739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:52.941771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fc98decc-a8ad-44ea-8ff9-2872832437f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049108 May 24 10:24:52.942353 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 596/1160] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:53.287077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:53.287794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:53.294242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:53.294684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:53.295436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:53.295865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:53.423752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:53.432123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-27e94ef0-c35d-4fbe-a4ef-3d3db209d5ae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148987 May 24 10:24:53.432796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 565/1161] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:53.953743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:53.954652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:53.962494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:53.962940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:53.963703 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:53.964128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:53.993683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7b65cdc0-5c02-4c95-80fb-4e0138808d16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044241 May 24 10:24:53.994228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 597/1162] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:24:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:54.449388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:54.449994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:54.456467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:54.456906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:54.457692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:54.458118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:54.587350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:54.595178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-55e7a4a8-0ca6-4d40-bb84-2be1889cf382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149650 May 24 10:24:54.595773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 566/1163] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:55.005421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:24:55.006924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:55.014132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:55.014575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:55.015322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:55.015750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:55.107839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0086676b-5ada-4405-a73b-22e314ddd7cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/action" status: 200 len: 76654 microversion: 2.1 time: 0.106058 May 24 10:24:55.108495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 598/1164] 10.222.0.35 () {60 vars in 1277 bytes} [Tue May 24 10:24:55 2022] POST /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1/action => generated 76654 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:24:55.612207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:55.612812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:55.616242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=58 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:55.620950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:55.621436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:55.622196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:55.622623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:55.761885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:55.765253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8d66217c-e1db-4db3-8f70-8e34e17f7a38 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.156916 May 24 10:24:55.765850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 567/1165] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:56.011888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:56.012472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:56.019657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.019941 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.020705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.021137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.053484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:56.053989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:56.060029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.060484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.061258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.061719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.065683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:56.121536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:24:56.354216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-503cff32-66a9-48f9-9d60-26f22ddf587b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 204 len: 0 microversion: 2.1 time: 0.304562 May 24 10:24:56.354822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 568/1166] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:24:56 2022] DELETE /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 24 10:24:56.363143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:56.363818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:56.369785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.370223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.370982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.371406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.502031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:56.510281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9948f8ba-7b9f-4d57-8801-82d3d064dc09 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1673 microversion: 2.1 time: 0.151076 May 24 10:24:56.510864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 569/1167] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:56 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1673 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:56.783464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:56.784095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:56.790924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.791377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:56.792147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:56.792584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:57.013810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:57.021925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6f508e67-cf18-4bc1-b9f5-d849ad6de990 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.242416 May 24 10:24:57.022508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 570/1168] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:57.546349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:57.546349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:57.546349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:57.546349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:57.546959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:57.546959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:57.630436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:57.638923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4e46c278-72b2-4281-bf9e-4f7176f8256b tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1605 microversion: 2.1 time: 0.113736 May 24 10:24:57.639551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 571/1169] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:57 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1605 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:24:57.884843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-75d48084-ca77-4b4f-a137-20ba16d63404 req-4e811cfb-258d-4a96-ae32-728f311ccdd3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d6651fff-37e6-4e58-9f84-5e58f2579414", "name": "network-vif-deleted", "tag": "313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:24:57.894831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-75d48084-ca77-4b4f-a137-20ba16d63404 req-4e811cfb-258d-4a96-ae32-728f311ccdd3 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:57.895401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-75d48084-ca77-4b4f-a137-20ba16d63404 req-4e811cfb-258d-4a96-ae32-728f311ccdd3 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:57.909478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-75d48084-ca77-4b4f-a137-20ba16d63404 req-4e811cfb-258d-4a96-ae32-728f311ccdd3 service nova] Creating event network-vif-deleted:313723cf-6ab4-4fb4-89ef-b6a6c5dd5b3f for instance d6651fff-37e6-4e58-9f84-5e58f2579414 on hv-cinder-84311 May 24 10:24:57.915318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-75d48084-ca77-4b4f-a137-20ba16d63404 req-4e811cfb-258d-4a96-ae32-728f311ccdd3 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099879 May 24 10:24:57.915917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 572/1170] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:24:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:24:58.040216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:58.040919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:58.047259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:58.047713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:58.048490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:58.048930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:58.197302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:58.205036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e4fbe360-ec4e-455b-8bff-c0e9e6949ede tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.169225 May 24 10:24:58.205860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 573/1171] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:58 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:58.656658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:58.657292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:58.664573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:58.665136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:58.665974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:58.666449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:58.702533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:58.704207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:58.705061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lazy-loading 'fault' on Instance uuid d6651fff-37e6-4e58-9f84-5e58f2579414 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:24:58.755705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7bffd469507b4dbc8d3146f051c8c8ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:58.764036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6673b380-8f9b-4466-9d1f-632ac29b85a8 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 200 len: 1328 microversion: 2.1 time: 0.111414 May 24 10:24:58.764820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 574/1172] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:58 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 1328 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:59.221610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:59.222245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:59.228934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:59.229929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:59.237141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:59.237141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:59.291002 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lazy-loading 'services' on Instance uuid c4942f39-67fa-4c51-b915-7354d2c78f00 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:24:59.384807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:24:59.393000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e37d3bf9-78a8-4518-8c3d-ed5ab7bb0d8e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.175348 May 24 10:24:59.393615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 575/1173] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:24:59.781844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:59.782482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:24:59.788465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:59.788922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:59.789782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:24:59.790211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:24:59.806857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] HTTP exception thrown: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Request method failure captured: May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 HTTP/1.1 May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_TJiJC50rIjdcS63u7xuYhM4UlnZwxRv8mUKp0BzcDusYujaMOjfiQ7B-7czQB8mmEz41Y8c7oBuVaj_T84-448VkgY0Pf5gNrx-P1niD879m3w66ryHsb9xDN21s3gpWLUkY4irhyPqPHyYMccDT3Xqb0fUHM9qSZRWOI7k2kkeY May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:24:59.808491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeTestJSON-1912151979 May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 7bffd469507b4dbc8d3146f051c8c8ad May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:24:59.810092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'd6651fff-37e6-4e58-9f84-5e58f2579414'} May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:24:59.810993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:24:59.812481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:24:59.813837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Returning 404 to user: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:24:59.813837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-29750149-60fe-44c2-9083-a499b590bb17 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 404 len: 111 microversion: 2.1 time: 0.031980 May 24 10:24:59.813837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 576/1174] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:24:59 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:24:59.818462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2b4c712f-923a-4d4e-846a-b34b9382fae6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '.version_select at 0x7f873252a790>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:24:59.892134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2b4c712f-923a-4d4e-846a-b34b9382fae6 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1169943927" status: 202 len: 0 microversion: 2.1 time: 0.077069 May 24 10:24:59.892698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 577/1175] 10.222.0.35 () {58 vars in 1220 bytes} [Tue May 24 10:24:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1169943927 => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:25:00.411376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:00.412065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:00.419622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:00.420083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:00.420853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:00.421321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:00.555109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:00.563145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-49e2c2e8-3810-4d99-940b-761b192ab99d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.156230 May 24 10:25:00.563728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 578/1176] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:25:01.580708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:01.581387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:01.588063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:01.588505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:01.589281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:01.589714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:01.719076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:01.726661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a5847f4-6ebe-4e29-903f-7ba65680c7c0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149761 May 24 10:25:01.727951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 579/1177] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:02.747303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:02.747922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:02.754427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:02.754877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:02.755666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:02.756103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:02.884339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:02.892021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-701d111f-5a31-4877-a78a-f239b6679949 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151185 May 24 10:25:02.892587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 580/1178] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:03.909144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:03.909857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:03.916731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:03.917183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:03.917972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:03.918409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:04.050157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:04.058529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-28b741c8-c899-43e7-ab6c-97ad56054110 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153384 May 24 10:25:04.059079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 581/1179] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:05.076187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:05.076824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:05.083389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:05.083836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:05.084598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:05.085030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:05.218307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:05.225874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3dccda07-29e0-4351-aefc-fed4a4a3a415 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154130 May 24 10:25:05.226444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 582/1180] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:06.016886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:06.243292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:06.244333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:06.250922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:06.251366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:06.252131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:06.252566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:06.389872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:06.404247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-53306cf1-c783-40bd-b97d-920b2a5365f5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.157842 May 24 10:25:06.404247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 583/1181] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:07.413902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:07.414547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:07.421690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:07.422080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:07.422835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:07.423267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:07.552587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:07.561166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-351e6e3b-8f32-4acc-bb89-4c98ec8795b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151070 May 24 10:25:07.561760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 584/1182] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:08.582674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:08.583316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:08.590195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:08.590697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:08.591498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:08.592007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:08.596166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:08.728053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:08.735620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b94ff2e0-2f9f-4d44-ad45-b8a2cef5c83a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.159420 May 24 10:25:08.736103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 585/1183] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:25:09.334819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=26 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:09.753770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:09.754359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:09.762184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:09.762632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:09.763400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:09.763833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:09.893210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:09.901419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b0ddd988-ffdc-4209-9bd2-405c0722137e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151905 May 24 10:25:09.901991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 586/1184] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:10.918388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:10.919044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:10.925601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:10.926050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:10.926852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:10.927241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:11.054755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:11.062903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ad1e8c98-b3f1-46ed-9c10-73e15e1dd7d5 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148522 May 24 10:25:11.063481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 587/1185] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:12.080161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:12.080868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:12.087887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:12.088424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:12.089189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:12.089641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:12.221592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:12.229061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6e8b012c-d4a3-4721-8890-99322e6c5638 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152749 May 24 10:25:12.229661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 588/1186] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:13.246487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:13.247199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:13.253511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:13.253954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:13.254705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:13.255132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:13.385567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:13.393364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4593c8b5-c920-42b9-9827-530f603cc8c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151289 May 24 10:25:13.393938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 589/1187] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:14.411890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:14.412547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:14.419274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:14.419658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:14.420410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:14.420834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:14.551939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:14.561292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fc676bac-fba3-4f9a-832b-4da09eb22f05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153202 May 24 10:25:14.561869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 590/1188] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:15.579029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:15.579708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:15.586049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:15.586502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:15.587257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:15.587684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:15.716958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:15.724410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd79a6a0-3029-4640-83bb-dcc2c3598907 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149216 May 24 10:25:15.724971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 591/1189] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:16.741543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:16.742152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:16.749439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:16.749889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:16.750648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:16.751102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:16.886075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:16.893537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0695e4ce-d22b-41e4-8ccf-9ed0852f2bc6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.155838 May 24 10:25:16.894112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 592/1190] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:17.910816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:17.911482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:17.918295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:17.918749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:17.919517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:17.919955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:18.052812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:18.060589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3b0d0a21-a7c5-4ab7-bebb-7a069f8c7809 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153672 May 24 10:25:18.061151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 593/1191] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:19.077465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:19.078109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:19.084501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:19.084953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:19.085797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:19.086232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:19.218275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:19.225936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a75326c6-5714-4d19-858b-abff103a4728 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152492 May 24 10:25:19.226510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 594/1192] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:20.231432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:25:20.231944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:25:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d (ip 10.222.0.35) !!! May 24 10:25:20.243279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:20.243883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:20.251173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:20.251616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:20.252399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:20.252826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:20.388851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:20.398334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-96d31113-6f6e-479f-8030-45ba424b5cc4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.158259 May 24 10:25:20.399068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 595/1193] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:21.416613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:21.417286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:21.423753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:21.424201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:21.424961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:21.425439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:21.557065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:21.565571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6fd27b9c-f4de-4968-a2e3-d2ca9b987ab6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152953 May 24 10:25:21.566183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 596/1194] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:22.582784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:22.583471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:22.590609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:22.591050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:22.591807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:22.592232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:22.722823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:22.730241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-11fe4eb2-cfb8-4c56-b1a4-fc1a68d250ba tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151345 May 24 10:25:22.730807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 597/1195] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:23.747198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:23.747861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:23.754216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:23.754660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:23.755411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:23.755843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:23.883909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:23.892027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b5aa148b-e958-4057-9dd7-d9a125755627 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148909 May 24 10:25:23.892588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 598/1196] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:24.909114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:24.909830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:24.916630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:24.917873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:24.918899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:24.919342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:25.048431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:25.056274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4990a9aa-2f1b-4b28-abea-ad918a470f09 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151145 May 24 10:25:25.056834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 599/1197] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:26.173128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:26.173786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:26.175625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=74,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:26.180574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:26.181011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:26.181803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:26.182229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:26.289258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:25:26.289514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:25:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b (ip 10.222.0.35) !!! May 24 10:25:26.315243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:26.323999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f8644bc3-ae13-4359-9196-2e318fd55422 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.254539 May 24 10:25:26.324583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 600/1198] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:27.340950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:27.341659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:27.348072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:27.348520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:27.349342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:27.349777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:27.478196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:27.482171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:27.486583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d86c2512-c887-4561-a577-ba836303af5b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149582 May 24 10:25:27.490511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 601/1199] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:28.504976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:28.505697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:28.512772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:28.513212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:28.514010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:28.514439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:28.648705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:28.656454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bde20ecd-726c-49f2-a11e-e8ef55b1426a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.155673 May 24 10:25:28.657104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 602/1200] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:29.673459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:29.674115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:29.680799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:29.681238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:29.682009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:29.682441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:29.811835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:29.819210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fb9c1b6d-822d-42ce-883e-0b77a64816cf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149631 May 24 10:25:29.819771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 603/1201] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:30.835947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:30.836605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:30.843080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:30.843521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:30.844271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:30.844699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:30.975109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:30.983812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-97207cac-2895-4873-9373-a40970184488 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151702 May 24 10:25:30.984380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 604/1202] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:32.001857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:32.002532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:32.009067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:32.009546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:32.010329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:32.010745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:32.140822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:32.148370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-704ac1f4-7f21-4379-88d1-813ed9bb8566 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150371 May 24 10:25:32.148941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 605/1203] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:33.165473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:33.166077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:33.172698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:33.173141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:33.173927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:33.174356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:33.301977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:33.310482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-57a8a131-9bcb-431d-aa95-9a8631ec7ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148910 May 24 10:25:33.311061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 606/1204] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:25:34.328220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:34.328882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:34.335293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:34.335740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:34.336527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:34.336963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:34.468647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:34.476061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-55f49d14-30ea-4991-aa9f-9733d1657830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151772 May 24 10:25:34.476624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 607/1205] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:35.493055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:35.493749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:35.500173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:35.500617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:35.501405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:35.501834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:35.629466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:35.636905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-060ae1be-d65f-4494-9d3b-ad6a57f96678 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.147634 May 24 10:25:35.637494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 608/1206] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:36.653853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:36.654558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:36.660878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:36.661344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:36.662100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:36.662526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:36.797131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:36.804835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-342ba60a-3941-4682-a624-070027cc774c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154780 May 24 10:25:36.805455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 609/1207] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:37.822745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:37.823470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:37.830743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:37.831195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:37.831970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:37.832444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:37.962576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:37.970256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7290a62a-1753-4dde-ab5c-ec72065c676d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151461 May 24 10:25:37.970817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 610/1208] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:38.987141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:38.987843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:38.994020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:38.994471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:38.995253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:38.995715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:39.207914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:39.216312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-65b1bdcc-72c7-44f6-9c84-d79dcc3ba944 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.232435 May 24 10:25:39.216926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 611/1209] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:40.241767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:40.242407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:40.249814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:40.250263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:40.251031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:40.251467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:40.387797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:40.395758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d63e0524-fc59-45d7-8b0a-4ec6ac7e2fa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.157836 May 24 10:25:40.396340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 612/1210] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:41.412995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:41.414240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:41.420461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:41.420850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:41.421650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:41.422085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:41.551094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:41.558807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-64e7cb5a-8214-4c00-90c9-7df5e16733b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149694 May 24 10:25:41.559900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 613/1211] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:25:42.576600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:42.577257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:42.586061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:42.586536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:42.587320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:42.587758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:42.724547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:42.732213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-37852810-0edf-4c00-b880-6f55d3d1b896 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.160172 May 24 10:25:42.732782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 614/1212] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:43.749682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:43.750300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:43.756692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:43.757139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:43.757967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:43.758407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:43.886146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:43.893766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fad16113-9b20-4e84-a90c-1cf94ca93a5e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148628 May 24 10:25:43.894330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 615/1213] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:44.910988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:44.911651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:44.918038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:44.918481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:44.919240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:44.919714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:45.049893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:45.057752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7831133d-0529-455a-bda5-a9eaa94d9c2f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150639 May 24 10:25:45.058312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 616/1214] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:46.075419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:46.076177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:46.082717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:46.083164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:46.083915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:46.084342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:46.215842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:46.223911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-329d87eb-91d1-4357-8887-40586e1731d2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152415 May 24 10:25:46.224476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 617/1215] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:47.240673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:47.241319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:47.247819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:47.248868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:47.249664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:47.250098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:47.267640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:47.377250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:47.384801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f96c10fb-8a0b-461c-b741-a1645768ebe8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.147950 May 24 10:25:47.385408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 618/1216] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:48.401585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:48.402223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:48.408885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:48.409364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:48.410125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:48.410564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:48.540983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:48.549235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2195be6e-9421-4cfb-8216-2df8aaa391b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151540 May 24 10:25:48.549840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 619/1217] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:49.567811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:49.568465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:49.574936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:49.575385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:49.576146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:49.577456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:49.705866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:49.713422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-19ee30c0-a036-438f-9759-4d363b983c6f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149477 May 24 10:25:49.713987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 620/1218] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:50.730838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:50.731447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:50.737933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:50.738383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:50.739145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:50.739579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:50.869195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:50.876783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ca8be4b5-e3f7-4cd6-993e-903678742817 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149780 May 24 10:25:50.877369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 621/1219] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:25:51.892951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:51.893720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:51.900092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:51.900539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:51.901328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:51.901764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:52.040967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:52.040967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eca66c43-697b-4bd5-9eb4-cd5bddcaa2ed tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151629 May 24 10:25:52.041219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 622/1220] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:53.058605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:53.059211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:53.065846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:53.066292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:53.067048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:53.067483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:53.196244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:53.204188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e5f87b5a-1b36-4525-a63d-03e2e8e44d1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149788 May 24 10:25:53.204757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 623/1221] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:54.221959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:54.222598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:54.229043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:54.229534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:54.230322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:54.230759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:54.364616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:54.372066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-93ba5040-c956-4cb9-a00b-51543a524119 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154805 May 24 10:25:54.372636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 624/1222] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:55.390808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:55.391535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:55.398084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:55.398542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:55.399307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:55.399765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:55.530813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:25:55.538338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-16896703-8e3c-4a0f-a3b4-ecdbf4d7c4bf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152710 May 24 10:25:55.538909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 625/1223] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:25:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:25:56.140897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:25:56.141420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:25:56.147452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:56.147893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:25:56.148652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:25:56.149079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:06.145527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=53 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:26:06.167834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=48 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:26:53.091891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:26:53.092454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:26:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d (ip 10.222.0.35) !!! May 24 10:26:56.388758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0aa39a50-5a77-40f9-95d4-12be662b51a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d" status: 202 len: 0 microversion: 2.1 time: 120.380537 May 24 10:26:56.393970 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 599/1224] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:24:56 2022] DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d => generated 0 bytes in 120383 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:26:56.395804 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:26:56.396509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:26:56.404154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:26:56.404670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:56.405531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:26:56.406149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:56.474736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lazy-loading 'services' on Instance uuid c4942f39-67fa-4c51-b915-7354d2c78f00 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:26:56.553326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:26:56.556079 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-42d4b411-a46e-4a38-98b4-316af09cafdf tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.165205 May 24 10:26:56.556645 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 600/1225] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:26:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:26:56.562243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:26:56.562942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:26:56.569446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:26:56.569904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:56.570707 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:26:56.571168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:59.463751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Request method failure captured: May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d HTTP/1.1 May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLBVl3Urd-y_lJGU7mpRi910LZ9wevShzCRu_NwiSUYMpfxObJwflmsI4Of-vghkXEmbN7k4JVoMQJzBeU_ObpcHO1pW9yFA2YGSnXzr1eiYFxuZfPbxkO_2ZUypBbR3eooiJ-8kLYfYX1w55TYzcxHrfsTiB38kcF6Xl4psnIPY_Kw May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:26:59.471777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: bb2d87f281834e4697a8c3559b6e83a2 May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:26:59.473539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'server_id': 'c4942f39-67fa-4c51-b915-7354d2c78f00', 'id': '816a261e-61a2-4f02-931e-d7d498967c0d'} May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:26:59.474340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:26:59.475714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:26:59.477118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:26:59.478391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) May 24 10:26:59.479701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:26:59.481003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a6232053-b1cb-443c-bd37-507472cbf353) {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:26:59.481003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7e84d7bd-6670-4fce-8342-bea822fa66ce tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d" status: 400 len: 269 microversion: 2.1 time: 63.425786 May 24 10:26:59.481003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 626/1226] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:25:56 2022] DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d => generated 269 bytes in 63427 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 24 10:26:59.487901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:26:59.488523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:26:59.494597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:26:59.505736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:59.505736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:26:59.505736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:26:59.516945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lazy-loading 'services' on Instance uuid c4942f39-67fa-4c51-b915-7354d2c78f00 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:26:59.639821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:26:59.647684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e082ce92-0077-43f5-9e94-08c26f7dcc16 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.177206 May 24 10:26:59.648306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 627/1227] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:26:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:27:00.665304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:00.666004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:00.672961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:00.673441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:00.674209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:00.674646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:00.803595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:00.811844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b2eca1d9-cf7c-4b25-bbc4-928d592f1ade tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150496 May 24 10:27:00.812413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 628/1228] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:01.828802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:01.829575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:01.836096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:01.836546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:01.837331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:01.837767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:01.967749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:01.975194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-13aabcbf-4e8b-4660-96fb-5ade68e17e41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150936 May 24 10:27:01.975761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 629/1229] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:02.992536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:02.993345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:03.000327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:03.000776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:03.001569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:03.002016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:03.134582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:03.142696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4eed4fa5-e2b7-4bc5-8812-998f830b55de tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154048 May 24 10:27:03.143266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 630/1230] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:04.159590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:04.160265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:04.166971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:04.167424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:04.168198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:04.168636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:04.297834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:04.305343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1eaaf607-c807-4c60-806e-ba2cb73c6495 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149645 May 24 10:27:04.306208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 631/1231] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:05.322836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:05.323521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:05.329971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:05.330413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:05.331200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:05.331599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:05.462000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:05.469822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ac44c707-4815-42ce-af06-ca4423fcb698 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150961 May 24 10:27:05.470422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 632/1232] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:06.486105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:06.486757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:06.493313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:06.493772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:06.494542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:06.494983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:06.569520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=4 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:27:06.627276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:06.634934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-83549076-b26a-4e01-a9cb-8c3ff4914674 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152740 May 24 10:27:06.635510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 633/1233] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:07.652655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:07.653351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:07.660246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:07.660691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:07.662713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:07.663420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:07.795579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:07.803196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fccc8158-8c82-433a-a2db-1813bcc6b0c3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154369 May 24 10:27:07.803760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 634/1234] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:08.819925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:08.820629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:08.826920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:08.827363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:08.828121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:08.828545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:08.954742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:08.962237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-db8e9a19-966b-42bf-8892-3e32ff3e7597 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.146209 May 24 10:27:08.962812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 635/1235] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:09.521680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=15,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:27:09.888856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:27:09.889397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:27:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:27:09.979151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:09.979786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:09.986812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:09.987260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:09.988019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:09.988451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:10.118107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:10.126292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-23fa6212-d983-4cd2-b146-bb6d40b63b66 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150890 May 24 10:27:10.126908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 636/1236] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:11.144718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:11.145342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:11.151878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:11.152326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:11.153085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:11.153547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:11.284353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:11.291858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-72f8311a-75b7-45fc-9d3c-8480de1e59ee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151472 May 24 10:27:11.292429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 637/1237] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:12.309751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:12.310473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:12.316842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:12.317297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:12.318056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:12.318483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:12.449903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:12.458108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0af51955-ff93-4162-8144-8231639af6b1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152476 May 24 10:27:12.458673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 638/1238] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:13.476111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:13.476719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:13.483181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:13.484639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:13.484639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:13.484905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:13.616901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:13.624265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3b4e4ab1-7d6a-4259-a06d-ad94ceb1dc7e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152397 May 24 10:27:13.624827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 639/1239] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:14.641257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:14.641955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:14.648489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:14.648937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:14.649723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:14.650157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:14.780658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:14.789146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0f4a5598-83d7-40b5-b677-a34fb7bb5ffe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151869 May 24 10:27:14.789750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 640/1240] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:15.806683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:15.807384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:15.813740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:15.814184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:15.814961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:15.815453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:15.943385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:15.950861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d51a8131-2b7d-49bc-a8bc-cce67f5eb714 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148356 May 24 10:27:15.951437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 641/1241] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:16.967828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:16.968500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:16.975168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:16.975614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:16.976373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:16.976801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:17.109772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:17.113933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:27:17.118430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7d738b1c-eca9-4945-9938-bc2849a264d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154459 May 24 10:27:17.119002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 642/1242] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:18.139756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:18.140406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:18.147149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:18.147589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:18.148346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:18.148777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:18.277383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:18.285370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9368e555-75ae-4797-a95c-3addf95cee7c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149819 May 24 10:27:18.285942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 643/1243] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:19.303083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:19.303727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:19.310389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:19.310831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:19.311587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:19.312015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:19.442313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:19.449788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8bbd18a7-3d3d-4c1e-93c0-3e21bc175b53 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151153 May 24 10:27:19.450355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 644/1244] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:20.467580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:20.468232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:20.475648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:20.476099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:20.476855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:20.477300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:20.606410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:20.613914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ed3e7139-b598-4246-91f8-c1c5705b9fb9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150394 May 24 10:27:20.614488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 645/1245] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:21.631030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:21.631706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:21.638132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:21.638576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:21.639335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:21.639769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:21.773733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:21.781757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ac0f8216-377d-4f2f-9592-0b670bbef473 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154619 May 24 10:27:21.782456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 646/1246] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:22.799538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:22.800956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:22.807962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:22.808413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:22.809178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:22.809640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:22.936794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:22.944426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ced36bf0-ba33-43a0-8c2b-1b000289aa05 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148775 May 24 10:27:22.944986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 647/1247] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:23.960306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:23.961016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:23.967582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:23.968014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:23.968772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:23.969205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:24.098924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:24.106346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4df32acd-09b0-497b-afc1-56d11c2a280b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149959 May 24 10:27:24.106909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 648/1248] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:25.123940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:25.124587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:25.131659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:25.132114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:25.132880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:25.133412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:25.266062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:25.274353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-24d4bdfe-11e8-4077-9d2d-b29b22da4aea tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154388 May 24 10:27:25.274923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 649/1249] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:26.291554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:26.292217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:26.294454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=47 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:27:26.299196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:26.299644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:26.300561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:26.300975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:26.428026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:26.435933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e724c9aa-6d67-46c9-bb70-fcb6043c4ecd tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148207 May 24 10:27:26.436536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 650/1250] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:27.452989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:27.453648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:27.460332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:27.460781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:27.461573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:27.462012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:27.591918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:27.599846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0d626740-b0b0-494f-a43b-a251436a1dfb tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150692 May 24 10:27:27.600418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 651/1251] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:28.617699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:28.618200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:28.624350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:28.633566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:28.633566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:28.633566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:28.757557 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:28.765118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec9d3b73-414b-4b22-bcba-4bda2f4dc01b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151745 May 24 10:27:28.765738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 652/1252] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:29.782803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:29.783474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:29.790125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:29.791160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:29.791920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:29.792351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:29.924246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:29.932606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a01942e4-901b-4e9f-a4a6-13a4ab303220 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153628 May 24 10:27:29.933545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 653/1253] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:30.950494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:30.951091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:30.958097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:30.958544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:30.959306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:30.959738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:31.090920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:31.098748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8f716523-f372-4071-afe1-68852179bec8 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152153 May 24 10:27:31.099319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 654/1254] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:32.116171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:32.116819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:32.123250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:32.123695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:32.124454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:32.124883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:32.252718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:32.261102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-60f135e6-d985-4ddf-a41c-815a4a486aa6 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149388 May 24 10:27:32.261700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 655/1255] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:27:33.278718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:33.279397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:33.286604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:33.287050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:33.287814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:33.288244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:33.420073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:33.427731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a330b87-c326-4320-a38c-1d29ff61d5fa tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152948 May 24 10:27:33.428308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 656/1256] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:34.445384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:34.446066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:34.452488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:34.452937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:34.453759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:34.454191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:34.583139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:34.590820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cb4193a3-167a-4d87-ba10-43a45e17743b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149289 May 24 10:27:34.591397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 657/1257] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:35.608487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:35.609155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:35.616972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:35.617448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:35.618200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:35.618629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:35.749124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:35.756711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-26d8f599-ccdf-4221-901b-b1d10883cb32 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152157 May 24 10:27:35.757411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 658/1258] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:36.773943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:36.774592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:36.781127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:36.787409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:36.787409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:36.787409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:36.801711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:27:36.917093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:36.924528 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb2f43cd-6b31-4175-9a25-4c9e71f26260 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154591 May 24 10:27:36.925663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 659/1259] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:27:37.942097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:37.942744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:37.949135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:37.949608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:37.950361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:37.950788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:38.078847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:38.086950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c7fb72a2-a195-4982-bb06-e336e19d6a07 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.149440 May 24 10:27:38.087516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 660/1260] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:39.104281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:39.104885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:39.111019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:39.111462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:39.112216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:39.112849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:39.243509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:39.250922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a1395498-8531-4c7b-84a3-ccc8a23f699c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150624 May 24 10:27:39.251496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 661/1261] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:40.267793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:40.268424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:40.274879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:40.275326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:40.276104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:40.276518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:40.406479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:40.414799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b86fcd18-0f98-4235-a19b-4982a7f5dd99 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150811 May 24 10:27:40.415450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 662/1262] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:41.432395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:41.433032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:41.439737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:41.440181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:41.441649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:41.442182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:41.584120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:41.592470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-52c889af-750e-4674-9681-db9f94ed0dee tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.164431 May 24 10:27:41.593395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 663/1263] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:42.612075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:42.612932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:42.619421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:42.619807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:42.620566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:42.620991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:42.750156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:42.758124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-79d5d1a9-9e6d-43da-9953-9ed226ef55d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150051 May 24 10:27:42.758695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 664/1264] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:43.775677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:43.776350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:43.782724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:43.783171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:43.783928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:43.784363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:43.911936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:43.919449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e4a4f174-5977-42d8-82bc-58c5ec4b5f23 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.147704 May 24 10:27:43.920026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 665/1265] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:44.936488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:44.937111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:44.943724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:44.944166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:44.944965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:44.945407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:45.084108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:45.091766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-da1533a5-dba7-41c3-bbfb-4a9abdc22570 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.159488 May 24 10:27:45.092335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 666/1266] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:46.109416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:46.110113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:46.117182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:46.117654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:46.118417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:46.118857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:46.246670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:46.254185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c644162e-741b-4101-865b-c9fe6f226a87 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.148654 May 24 10:27:46.254756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 667/1267] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:47.271423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:47.272052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:47.278522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:47.278979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:47.279748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:47.280184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:47.414241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:47.421619 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a9bcf6e4-8fd3-474b-ae94-ec5164833a39 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.154413 May 24 10:27:47.422197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 668/1268] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:48.439889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:48.441045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:48.447946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:48.448392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:48.449154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:48.449618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:48.578145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:48.586485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-74587420-87ec-4cc2-94ee-15ad048f5097 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150408 May 24 10:27:48.587063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 669/1269] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:49.603931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:49.604598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:49.611192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:49.611641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:49.612426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:49.612891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:49.743422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:49.751118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-571410ba-4ad1-4e50-bc58-5f19dd216386 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.151114 May 24 10:27:49.751821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 670/1270] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:50.768031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:50.768858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:50.775096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:50.775543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:50.776309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:50.776736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:50.906708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:50.914756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8dc49d93-1abb-4b7c-b1e0-d3e4ad25569e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.150645 May 24 10:27:50.915326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 671/1271] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:51.932106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:51.932762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:51.939157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:51.939601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:51.940360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:51.940913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:52.078946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:52.087239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c3677a69-465a-4949-a643-639cda319cae tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.159258 May 24 10:27:52.087805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 672/1272] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:53.104118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:53.104714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:53.111145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:53.111571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:53.112335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:53.113368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:53.245848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:53.253404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c642a46d-c8cf-4f64-8017-c6f33b779fbc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.152983 May 24 10:27:53.254121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 673/1273] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:54.271439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:54.272222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:54.278722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:54.279165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:54.279930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:54.280487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:54.411914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:54.420365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2c2ca8c0-afe3-4f69-b5e5-45e820c5ab6b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153637 May 24 10:27:54.420878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 674/1274] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:27:55.437940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:55.438622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:55.445971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:55.446467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:55.447263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:55.447697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:55.579667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:27:55.587279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-15d16abd-9b37-4c7d-9d35-d901d91a0a54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.153389 May 24 10:27:55.587859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 675/1275] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:27:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:27:56.166077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:27:56.166699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:27:56.172730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:56.173182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:56.173979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:27:56.174416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:27:56.179022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:28:06.170501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=53 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:28:06.219484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:28:40.239638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:28:40.240197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:28:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d (ip 10.222.0.35) !!! May 24 10:28:46.299164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:28:46.299734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:28:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d (ip 10.222.0.35) !!! May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Request method failure captured: May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d HTTP/1.1 May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLBVl3Urd-y_lJGU7mpRi910LZ9wevShzCRu_NwiSUYMpfxObJwflmsI4Of-vghkXEmbN7k4JVoMQJzBeU_ObpcHO1pW9yFA2YGSnXzr1eiYFxuZfPbxkO_2ZUypBbR3eooiJ-8kLYfYX1w55TYzcxHrfsTiB38kcF6Xl4psnIPY_Kw May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:28:56.747005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: bb2d87f281834e4697a8c3559b6e83a2 May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:28:56.749156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'server_id': 'c4942f39-67fa-4c51-b915-7354d2c78f00', 'id': '816a261e-61a2-4f02-931e-d7d498967c0d'} May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:28:56.750008 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:28:56.751220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:56.752443 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:28:56.753509 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) May 24 10:28:56.754696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:56.755789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-356a0994-1e81-4e6e-9a51-927b09ea8eb0) {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:28:56.755789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-64e380be-90bc-49c3-bc7d-cccb8651f144 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d" status: 400 len: 269 microversion: 2.1 time: 120.185789 May 24 10:28:56.755789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 601/1276] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:26:56 2022] DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d => generated 269 bytes in 120187 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 24 10:28:56.755789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:28:56.756099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:28:56.757913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:28:56.758370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:28:56.759153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:28:56.759591 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:28:56.904285 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:28:56.912906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b389ccfe-a92d-4204-92e1-ff5d12e4d346 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.167855 May 24 10:28:56.913313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 602/1277] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:28:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:28:56.918148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:28:56.918655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:28:56.924822 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:28:56.925301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:28:56.926166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:28:56.926632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:28:57.446901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lazy-loading 'services' on Instance uuid c4942f39-67fa-4c51-b915-7354d2c78f00 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:28:57.640737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lazy-loading 'services' on Instance uuid c4942f39-67fa-4c51-b915-7354d2c78f00 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:28:59.733380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Request method failure captured: May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d HTTP/1.1 May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLBVl3Urd-y_lJGU7mpRi910LZ9wevShzCRu_NwiSUYMpfxObJwflmsI4Of-vghkXEmbN7k4JVoMQJzBeU_ObpcHO1pW9yFA2YGSnXzr1eiYFxuZfPbxkO_2ZUypBbR3eooiJ-8kLYfYX1w55TYzcxHrfsTiB38kcF6Xl4psnIPY_Kw May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:28:59.735644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: bb2d87f281834e4697a8c3559b6e83a2 May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:28:59.737750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'server_id': 'c4942f39-67fa-4c51-b915-7354d2c78f00', 'id': '816a261e-61a2-4f02-931e-d7d498967c0d'} May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:28:59.738589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:28:59.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.740843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:28:59.741907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) May 24 10:28:59.743025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:28:59.744083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-351a1b1c-a2ea-4545-b313-76d31e2f507f) {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:28:59.744083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb027105-4626-49a5-a1e4-d6b980e99bd3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d" status: 400 len: 269 microversion: 2.1 time: 63.575840 May 24 10:28:59.744083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 676/1278] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:27:56 2022] DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d => generated 269 bytes in 63577 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 24 10:28:59.744083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:28:59.745800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:28:59.753356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:28:59.753810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:28:59.756167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:28:59.756167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:28:59.774935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Request method failure captured: May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d HTTP/1.1 May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLBVl3Urd-y_lJGU7mpRi910LZ9wevShzCRu_NwiSUYMpfxObJwflmsI4Of-vghkXEmbN7k4JVoMQJzBeU_ObpcHO1pW9yFA2YGSnXzr1eiYFxuZfPbxkO_2ZUypBbR3eooiJ-8kLYfYX1w55TYzcxHrfsTiB38kcF6Xl4psnIPY_Kw May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:28:59.777153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: bb2d87f281834e4697a8c3559b6e83a2 May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:28:59.780850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'server_id': 'c4942f39-67fa-4c51-b915-7354d2c78f00', 'id': '816a261e-61a2-4f02-931e-d7d498967c0d'} May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:28:59.781935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) May 24 10:28:59.784566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.787422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:28:59.789438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) May 24 10:28:59.791628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:28:59.794910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-25952425-1150-4c7d-93f2-45312839080a) {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:28:59.794910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-999d8813-d217-48a2-9b79-5931ddec0a48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d" status: 400 len: 269 microversion: 2.1 time: 2.864341 May 24 10:28:59.794910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 603/1279] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:28:56 2022] DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00/os-volume_attachments/816a261e-61a2-4f02-931e-d7d498967c0d => generated 269 bytes in 2871 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) May 24 10:28:59.894831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:28:59.902819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b1a541fa-8e96-460d-85e2-7054f8859d8d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1446 microversion: 2.1 time: 0.163433 May 24 10:28:59.903408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 677/1280] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:28:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1446 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:00.446549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-569cad7c-01f5-4854-8d50-dcfdc0ac9cc2 req-bf930072-f277-4c84-b459-ad4dc865ec8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "426e8866-95a1-42e7-9e88-29f93c5e7d7b", "name": "network-vif-deleted", "tag": "0ed9cac0-6315-452a-9289-41b7a7958b3c"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:29:00.456705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-569cad7c-01f5-4854-8d50-dcfdc0ac9cc2 req-bf930072-f277-4c84-b459-ad4dc865ec8c service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:00.457238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-569cad7c-01f5-4854-8d50-dcfdc0ac9cc2 req-bf930072-f277-4c84-b459-ad4dc865ec8c service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:00.468616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-569cad7c-01f5-4854-8d50-dcfdc0ac9cc2 req-bf930072-f277-4c84-b459-ad4dc865ec8c service nova] Creating event network-vif-deleted:0ed9cac0-6315-452a-9289-41b7a7958b3c for instance 426e8866-95a1-42e7-9e88-29f93c5e7d7b on hv-cinder-84311 May 24 10:29:00.474520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-569cad7c-01f5-4854-8d50-dcfdc0ac9cc2 req-bf930072-f277-4c84-b459-ad4dc865ec8c service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032340 May 24 10:29:00.475130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 604/1281] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:29:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:29:00.919806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:00.920424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:00.927536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:00.928036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:00.928829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:00.929286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:00.993113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:01.000754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2f95c4c6-7d87-438b-910a-edf848226f0b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084881 May 24 10:29:01.001336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 678/1282] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:02.020157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:02.020856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:02.028948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:02.029431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:02.030198 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:02.030637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:02.091901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:02.099894 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1d338a0d-872f-4d37-a1c6-3599e3cbe745 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084754 May 24 10:29:02.100457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 605/1283] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:29:03.117183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:03.117745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:03.124139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:03.124562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:03.125341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:03.125771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:03.187530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:03.195729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f73417b2-105e-41fc-b9c9-62f92117a663 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082503 May 24 10:29:03.196301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 679/1284] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:29:04.212992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:04.213635 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:04.220243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:04.220683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:04.221815 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:04.222265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:04.283014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:04.290891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fe00f222-f56c-4af5-8d4b-78af0b9fbd8c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082049 May 24 10:29:04.291462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 606/1285] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:05.308089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:05.308946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:05.316213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:05.316661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:05.317450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:05.317879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:05.377167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:05.385412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-78a32f9b-b2f9-4dff-a2a6-218ed2b285e7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081226 May 24 10:29:05.388462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 680/1286] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:06.402512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:06.403129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:06.410069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:06.410524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:06.411292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:06.411738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:06.472744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:06.480269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9f834d45-38eb-4946-b10e-5c259e97383e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081623 May 24 10:29:06.480841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 607/1287] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:07.497392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:07.497998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:07.504608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:07.505053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:07.505846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:07.506282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:07.566235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:07.573686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5eac45c0-e984-440f-b7bc-4246d838593b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080165 May 24 10:29:07.574254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 681/1288] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:08.591016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:08.591621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:08.598670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:08.599116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:08.599868 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:08.600297 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:08.666211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:08.677492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5bb8b39e-d250-4455-8754-9278cafac0a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086424 May 24 10:29:08.677492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 608/1289] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:08 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:09.691318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:09.691839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:09.698126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:09.698569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:09.699325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:09.699749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:09.762485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:09.770263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-14fd57c6-112d-4f69-81a4-9bd97570b64c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083563 May 24 10:29:09.770835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 682/1290] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:10.787753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:10.788359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:10.794881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:10.795324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:10.796926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:10.796926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:10.858630 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:10.866959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8ec6ed0d-f449-4e25-b458-feb08118bb43 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083651 May 24 10:29:10.867521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 609/1291] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:11.883643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:11.884249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:11.890849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:11.891314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:11.892088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:11.892520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:11.955501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:11.963589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e1ca9509-3e46-4de8-8872-7f70ec3ed479 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084159 May 24 10:29:11.964154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 683/1292] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:12.980126 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:12.980732 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:12.987384 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:12.988477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:12.989363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:12.989800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:13.049138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:13.057114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-41edb83a-c910-4e66-ba99-cb610501fb02 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080799 May 24 10:29:13.057647 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 610/1293] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:14.074917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:14.075519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:14.082670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:14.083117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:14.083881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:14.084318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:14.145860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:14.153288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8992d428-2fd9-4799-b744-15940fd6b51c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082358 May 24 10:29:14.153867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 684/1294] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:15.170477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:15.171093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:15.178322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:15.178773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:15.179533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:15.179971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:15.240720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:15.248192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f02e3c7d-8dce-47d7-b06e-ffb0a495b555 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081671 May 24 10:29:15.248752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 611/1295] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:16.265048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:16.265701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:16.272217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:16.272663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:16.273450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:16.273883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:16.334276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:16.341661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef1c66ef-dee1-4a50-a392-d6255afc769d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080416 May 24 10:29:16.342219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 685/1296] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:17.359225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:17.359832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:17.366774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:17.367220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:17.367969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:17.368395 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:17.430986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:17.438757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-19f49d3e-4573-45bf-98fe-e45ca5e2884c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083434 May 24 10:29:17.439312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 612/1297] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:18.455951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:18.457237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:18.463727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:18.464173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:18.464934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:18.465389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:18.527898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:18.536026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-41cc2d23-c06c-4aa2-8e2b-43d01000c20a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083950 May 24 10:29:18.536581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 686/1298] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:19.552219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:19.552839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:19.559649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:19.560094 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:19.560865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:19.561366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:19.621559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:19.623808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=69 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:19.630295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-487d0fc4-67f2-4458-8fd7-0cc5be96983e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082428 May 24 10:29:19.630865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 613/1299] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:19 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:20.647968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:20.648558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:20.655244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:20.655691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:20.656481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:20.656920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:20.717861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:20.725480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c53e54e4-a916-4458-bb87-e1f0449d1454 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082091 May 24 10:29:20.726060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 687/1300] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:21.742622 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:21.743234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:21.750101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:21.750550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:21.751308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:21.751738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:21.811692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:21.819251 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e739ff49-f1b1-4ad9-bfb1-278092c3ab0c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080467 May 24 10:29:21.819813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 614/1301] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:22.836088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:22.836700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:22.843218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:22.843665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:22.844424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:22.844856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:22.904684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:22.912796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d2c4edbd-4205-4c1d-8d8b-6e1b1151abd9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080664 May 24 10:29:22.913386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 688/1302] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:23.930203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:23.930727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:23.937658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:23.938111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:23.938873 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:23.939308 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:24.001208 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:24.009613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-59f81b53-4e2d-4c0e-aa04-c743cd6f0af3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084003 May 24 10:29:24.010212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 615/1303] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:25.026937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:25.027543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:25.033983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:25.034427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:25.035869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:25.036318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:25.099201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:25.106640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a759d24e-3466-4ae8-9111-7b330529d5a1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083546 May 24 10:29:25.107321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 689/1304] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:26.123879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:26.124477 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:26.130964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:26.131434 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:26.132177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:26.132610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:26.198107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:26.205545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7dbf1885-e50a-4dba-b101-0f6eb88dca93 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085526 May 24 10:29:26.206099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 616/1305] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:27.223596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:27.224224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:27.226634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=27 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:27.231472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:27.231950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:27.232719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:27.233158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:27.237123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=83 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:27.297312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:27.305687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9d35b470-f45c-453e-872c-49241b8d661f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086347 May 24 10:29:27.306256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 690/1306] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:28.323832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:28.324423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:28.326715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=31 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:28.331426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:28.331865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:28.333249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:28.333698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:28.394319 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:28.404015 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f740f26e-e5b2-44fd-9e79-3c96ce4e7671 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084093 May 24 10:29:28.411612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 617/1307] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:29:29.421207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:29.421773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:29.428313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:29.428762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:29.429559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:29.429990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:29.490386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:29.497882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5b339925-5654-4f81-b3d5-1384e2f510b4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080985 May 24 10:29:29.498452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 691/1308] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:29 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:30.514957 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:30.515566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:30.522214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:30.522656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:30.523412 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:30.523876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:30.585088 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:30.592588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a02930a9-8722-46db-ab10-a8679d5b8ed1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081476 May 24 10:29:30.597466 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 618/1309] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:31.610066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:31.610768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:31.617895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:31.618341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:31.619106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:31.619538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:31.679639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:31.687402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9d73090b-7137-44f0-ac7f-91b484c188b2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081204 May 24 10:29:31.687975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 692/1310] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:32.704026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:32.704641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:32.711513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:32.711987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:32.712737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:32.713225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:32.900647 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:32.908130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23f6efd9-6f87-4b84-a8e2-1c807672c296 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.208063 May 24 10:29:32.908716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 619/1311] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:33.925028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:33.925670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:33.932185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:33.932637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:33.933432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:33.933868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:33.994097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:34.001533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-597fd555-6f2a-4861-8474-95a84edc5a4d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080362 May 24 10:29:34.002107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 693/1312] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:35.018728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:35.019363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:35.026215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:35.026665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:35.027428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:35.027865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:35.092618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:35.100765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7c14182f-fc83-4541-a853-d3852595ae08 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085877 May 24 10:29:35.101349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 620/1313] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:36.118199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:36.118822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:36.125256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:36.125726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:36.126477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:36.126926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:36.190060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:36.198397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0def1a15-3988-4d16-81e6-5247888e2987 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084572 May 24 10:29:36.198962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 694/1314] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:37.215532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:37.216137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:37.222738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:37.227188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:37.227188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:37.227188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:37.286168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:37.293721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6e638b3c-71ed-4b26-a315-b813880473d3 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082113 May 24 10:29:37.295879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 621/1315] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:38.311156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:38.311766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:38.318319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:38.318762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:38.319526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:38.319955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:38.379417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:38.387233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2ef7ea6e-e2d9-462c-beda-5c5fda93e4a2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080186 May 24 10:29:38.387807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 695/1316] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:39.404474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:39.405004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:39.412119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:39.412852 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:39.413649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:39.414093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:39.478533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:39.485984 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-95814b05-1baf-4952-aa1a-eed814de5c2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085454 May 24 10:29:39.486547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 622/1317] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:40.503606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:40.504221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:40.511312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:40.511808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:40.512579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:40.513005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:40.575286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:40.582899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2b33a9ff-2008-4e54-8fde-bca4e2bd6239 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083274 May 24 10:29:40.583477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 696/1318] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:40 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:41.599854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:41.600494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:41.606983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:41.607429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:41.608194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:41.608659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:41.669383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:41.676979 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1740371f-f2f4-4b53-b199-8f3d4f14057b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081010 May 24 10:29:41.677613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 623/1319] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:42.693865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:42.694463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:42.701056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:42.701533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:42.702291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:42.702719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:42.762668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:42.770132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-28461b8d-22d5-49b0-a56a-672ec74eb09a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080161 May 24 10:29:42.770701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 697/1320] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:43.788616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:43.789186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:43.796414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:43.796862 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:43.797755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:43.798189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:43.868685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:43.877516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-32e638c9-7499-45ce-b655-ad81d495ee54 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.093350 May 24 10:29:43.878164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 624/1321] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:44.898716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:44.899329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:44.905778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:44.906226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:44.906990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:44.907420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:44.970722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:44.978487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ab7e6989-bfed-4430-8573-58646adcbd3b tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084493 May 24 10:29:44.979059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 698/1322] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:45.995287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:45.996002 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:46.002446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:46.002895 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:46.003665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:46.004100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:46.065959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:46.074498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-51ca11f7-deb3-42bf-ba7f-bd2c4e7e2854 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083065 May 24 10:29:46.075086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 625/1323] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:47.091245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:47.091788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:47.098325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:47.098753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:47.100143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:47.100579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:47.162635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:47.170613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-908b8843-0ccf-4fc7-b67e-f5b89ca9ad2d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083316 May 24 10:29:47.171215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 699/1324] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:48.188355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:48.189007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:48.200665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:48.201116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:48.201931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:48.202374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:48.274194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:48.281884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ade3fcfc-df9e-4779-b6cd-ae47b063c382 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.098286 May 24 10:29:48.282459 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 626/1325] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:48 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:49.298177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:49.298781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:49.305412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:49.305857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:49.306610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:49.307036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:49.366762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:49.374466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d88cd57d-fe0c-4178-a142-83f606dc3113 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080136 May 24 10:29:49.375064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 700/1326] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:49 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:50.391856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:50.392454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:50.399793 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:50.400246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:50.401014 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:50.401488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:50.463628 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:50.470864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-306ed3c2-7c93-4bb2-b25b-ef1892cb38d7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082907 May 24 10:29:50.471431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 627/1327] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:50 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:51.488503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:51.489119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:51.496573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:51.497020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:51.497810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:51.498245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:51.558859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:51.566663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-62f4f6af-ef49-4de0-be42-a9c4d6042e7a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082024 May 24 10:29:51.567233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 701/1328] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:51 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:52.583644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:52.584250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:52.590883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:52.591327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:52.592077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:52.592503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:52.596350 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=86 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:52.658301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:52.665779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-89e9ac0d-3d1c-4e44-980f-8f4c24fd6f44 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086066 May 24 10:29:52.666351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 628/1329] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:52 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:53.681800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:53.682417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:53.689286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:53.689739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:53.690495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:53.690929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:53.756327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:53.764398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0a82fbab-66f3-4f11-9eb9-e9f943c575b0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086601 May 24 10:29:53.764973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 702/1330] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:53 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:29:54.781780 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:54.782383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:54.788763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:54.789854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:54.790602 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:54.791032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:54.852380 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:54.860027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4272aaf3-7cbc-4369-a27a-dc7662ed5dd0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082114 May 24 10:29:54.860592 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 629/1331] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:54 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:55.878063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:55.878665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:55.885260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:55.885732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:55.886485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:55.886919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:55.946680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:55.954189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ad4af4a4-8c7c-4e4f-9912-e6aa8fbd61af tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080050 May 24 10:29:55.954775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 703/1332] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:55 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:56.970294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:56.970907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:56.977440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:56.977888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:56.978650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:56.979082 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:57.039697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:57.047287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3d5f6e95-fe47-4cf8-a303-e259f92c3ad7 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080861 May 24 10:29:57.047845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 630/1333] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:56 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:58.065232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:58.065892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:58.068108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=28 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:58.073558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:58.074005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:58.074768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:58.075193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:58.137129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:58.145307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6cc0af45-d1bd-4118-8396-28579116dafc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083988 May 24 10:29:58.145887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 704/1334] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:58 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:29:59.163141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:29:59.163794 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:29:59.167240 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=28 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:59.172002 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:59.172472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:59.173238 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:29:59.173706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:29:59.236329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:29:59.243874 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-faa1521c-f020-41fa-85ae-e99fa3945062 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084769 May 24 10:29:59.244440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 631/1335] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:29:59 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:00.259779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:00.260413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:00.267480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:00.267920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:00.268666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:00.269090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:00.271598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=90 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:00.331545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:00.339109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2f68be57-f5d3-4a07-8a39-e164fcb659df tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083202 May 24 10:30:00.339675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 705/1336] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:00 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:01.357060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:01.357869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:01.364589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:01.365032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:01.365826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:01.366261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:01.429538 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:01.437535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aa183d13-490e-4db3-b217-a2c2d0cb7a26 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084543 May 24 10:30:01.438098 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 632/1337] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:01 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:02.454737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:02.455353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:02.461851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:02.462294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:02.463043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:02.463469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:02.529555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:02.537168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8cdcbd8b-449b-49ba-8316-bfd3ddefae9e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086326 May 24 10:30:02.537779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 706/1338] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:02 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:03.554889 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:03.555505 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:03.562321 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:03.562771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:03.563534 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:03.563975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:03.626312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:03.633787 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c09d0b0c-71e3-4834-92db-5b362ff52187 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082917 May 24 10:30:03.634347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 633/1339] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:03 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:04.651469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:04.652069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:04.658683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:04.659141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:04.659926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:04.660359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:04.722058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:04.729566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-50437bd8-d8f0-427f-a701-7ad8e3805323 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081914 May 24 10:30:04.730131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 707/1340] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:04 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:05.746867 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:05.747404 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:05.753959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:05.754409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:05.755891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:05.755891 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:05.817493 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:05.824907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6e96a40a-ab06-4909-88b3-dea1a9e7f258 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082025 May 24 10:30:05.825497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 634/1341] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:05 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:06.841180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:06.841815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:06.848534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:06.848990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:06.849786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:06.850216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:06.919896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:06.928309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b2a4b3a7-0149-4615-af70-a4711298a5ab tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.091005 May 24 10:30:06.928872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 708/1342] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:06 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:07.945710 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:07.946237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:07.952625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:07.953072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:07.954472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:07.954931 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:08.014890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:08.023075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-af161250-b32f-4ec3-be9f-c43cc3e671da tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081344 May 24 10:30:08.023642 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 635/1343] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:07 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:30:09.040041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:09.040647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:09.046630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:09.047073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:09.048044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:09.048499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:09.112938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:09.120559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-671495ab-c1b9-4ed7-bc5f-3574cc8e57cc tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084548 May 24 10:30:09.121123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 709/1344] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:09 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:10.137322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:10.137933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:10.145181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:10.145649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:10.146426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:10.146860 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:10.212944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:10.220406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b1e1e299-b9b1-47c5-a477-50b3408f2ed0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086945 May 24 10:30:10.221743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 636/1345] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:10 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:11.238744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:11.239466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:11.245980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:11.246419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:11.247173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:11.247600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:11.311748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:11.322044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b42eb42a-c187-4fa7-8ddd-34e87256c3d0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.087778 May 24 10:30:11.322609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 710/1346] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:11 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:12.339125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:12.339906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:12.346576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:12.347102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:12.348218 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:12.348667 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:12.414361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:12.422042 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2395c1d-456c-402f-905b-4b67b2d7e195 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086738 May 24 10:30:12.422545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 637/1347] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:12 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:13.100000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:30:13.100203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:30:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:30:13.439325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:13.439970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:13.446937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:13.447386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:13.448156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:13.448585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:13.514085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:13.522222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-73480638-fd49-4c0c-9543-9edbce34d396 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.087011 May 24 10:30:13.523082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 711/1348] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:13 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:30:14.538812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:14.539423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:14.546006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:14.546457 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:14.547217 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:14.547651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:14.609368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:14.617436 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b4b48271-0857-4233-9078-1e4800c10a1c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082804 May 24 10:30:14.617995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 638/1349] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:14 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:15.635380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:15.636013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:15.643132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:15.643624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:15.644437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:15.644890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:15.706680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:15.714604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dfcaa9ed-4da6-4d5c-996a-55d5903815c1 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084300 May 24 10:30:15.723492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 712/1350] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:15 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:16.732153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:16.732766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:16.739336 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:16.739786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:16.740604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:16.741040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:16.801576 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:16.808960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7f8ffe77-f61c-4b58-a2ba-1beaea34b857 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080700 May 24 10:30:16.809564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 639/1351] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:16 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:17.826509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:17.827113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:17.833590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:17.834034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:17.834883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:17.835316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:17.894674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:17.902674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-73d7fe38-3b3d-4030-80aa-bf645e9ad1c9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080018 May 24 10:30:17.903238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 713/1352] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:17 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:18.918917 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:18.919479 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:18.925995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:18.926440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:18.927200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:18.927629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:18.992965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:19.000486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8293993e-2e68-4db9-9669-4d7695f0b17c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085529 May 24 10:30:19.001045 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 640/1353] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:18 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:20.018364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:20.018974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:20.025928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:20.026379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:20.027148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:20.027587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:20.092188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:20.100419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cf6689f9-eabf-4a04-80f5-8992bf88b511 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085256 May 24 10:30:20.101189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 714/1354] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:20 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:21.118608 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:21.119144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:21.125864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:21.126357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:21.127146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:21.127581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:21.189594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:21.197964 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-89e2e92a-b44f-4490-a26a-3223d3f800a9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083907 May 24 10:30:21.198538 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 641/1355] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:21 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:22.214590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:22.215194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:22.221735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:22.222185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:22.222947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:22.223381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:22.286400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:22.294701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-915683a9-0968-4bea-813a-1aeee0834478 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083949 May 24 10:30:22.295266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 715/1356] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:22 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:23.312138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:23.312743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:23.320017 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:23.320460 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:23.321216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:23.321683 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:23.325541 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=84 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:23.385411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:23.392933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a69ab937-7655-4adb-a0bc-74a9057b1524 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084935 May 24 10:30:23.393522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 642/1357] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:23 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:24.410083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:24.410738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:24.417719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:24.418116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:24.418881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:24.419313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:24.479210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:24.486695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-441929c5-07fe-4416-ac53-24a86a20cc74 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.080498 May 24 10:30:24.487266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 716/1358] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:24 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:25.503706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:25.504307 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:25.510821 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:25.511269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:25.512020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:25.512451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:25.575697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:25.583650 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6a1970bc-73ce-45c2-9882-fa3888db1d9f tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083743 May 24 10:30:25.584790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 643/1359] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:25 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:26.711860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:26.712483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:26.718656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:26.719109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:26.719882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:26.720324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:26.781971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:26.790342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d6ac08b0-5168-4930-a1b6-ea0e3497b518 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.192993 May 24 10:30:26.790912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 717/1360] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:26 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:27.807826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:27.808435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:27.815159 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:27.815601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:27.816352 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:27.816782 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:27.886159 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:27.894185 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8c889214-1e89-4217-abe2-9516deb9b496 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.090210 May 24 10:30:27.894750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 644/1361] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:27 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:28.911972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:28.913338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:28.915633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=28 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:28.920611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:28.921070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:28.922089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:28.922527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:28.989300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:28.998516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-466740a8-c2d1-4387-bd23-ce2c5ab4aa36 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.090385 May 24 10:30:28.999096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 718/1362] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:28 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:29.897387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:30:29.897638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:30:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:30:30.015440 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:30.016161 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:30.018586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=28 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:30.024201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:30.024788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:30.025641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:30.026469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:30.090834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:30.098583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2ac29806-07eb-4ec1-99ed-9e888f9997d4 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.087003 May 24 10:30:30.099157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 645/1363] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:30 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:31.116861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:31.117634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:31.124448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:31.124894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:31.125702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:31.126140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:31.128721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=84 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:31.192921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:31.201094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f71da6cd-3425-42ec-b4d1-a37750b32b41 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.088850 May 24 10:30:31.201812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 719/1364] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:31 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:32.218767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:32.219382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:32.227045 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:32.227632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:32.228410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:32.228842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:32.293163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:32.300990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-749f5eae-6f67-4bb1-ac07-de9733824e2e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.086616 May 24 10:30:32.301669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 646/1365] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:32 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:33.319008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:33.319618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:33.326425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:33.326876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:33.327661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:33.328099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:33.391302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:33.399934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4dfc9ea4-a3e2-481d-90a6-c1b63b7b6193 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084857 May 24 10:30:33.400507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 720/1366] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:33 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:34.416991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:34.417598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:34.424124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:34.424546 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:34.425326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:34.425760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:34.491162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:34.498733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6c6ad697-80bb-4069-88be-1ce4d2e691f0 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085776 May 24 10:30:34.499295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 647/1367] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:34 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:35.516082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:35.516740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:35.523751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:35.524209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:35.524983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:35.525479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:35.587155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:35.595958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ce5b76ae-9029-4dae-b11c-7723c01daf0a tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.083789 May 24 10:30:35.596532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 721/1368] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:35 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:36.613339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:36.613949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:36.621035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:36.621511 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:36.622270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:36.622713 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:36.689986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:36.698243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2494d82-7586-49f4-b30a-3dd47fd19759 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.088851 May 24 10:30:36.698800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 648/1369] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:36 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:37.385347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:30:37.385865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:30:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:30:37.715800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:37.716436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:37.723049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:37.723496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:37.724354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:37.725551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:37.793794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:37.801198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4be7070f-824d-4e1a-bd4c-005f3589f830 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.089269 May 24 10:30:37.801789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 722/1370] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:37 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:38.818765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:38.819475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:38.826249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:38.826700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:38.827469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:38.827903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:38.891932 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:38.900084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0528fe47-d215-4c7d-a3f3-9810f9e7223c tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085392 May 24 10:30:38.900689 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 649/1371] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:38 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:39.917581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:39.918130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:39.924091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:39.924542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:39.925322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:39.925759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:39.986398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:39.994925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ce0daca2-5956-49e2-813f-bb1cec7b174d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081487 May 24 10:30:39.995492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 723/1372] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:39 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:41.012429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:41.013048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:41.019598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:41.020051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:41.020811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:41.021289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:41.082056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:41.089705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4773b1d2-7ae6-436a-af19-b29653a26890 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081601 May 24 10:30:41.090263 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 650/1373] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:41 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:42.106805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:42.107414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:42.113918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:42.114361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:42.115118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:42.115551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:42.176969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:42.184527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cb2b1827-4b6f-4219-a03f-dd56c89c095d tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.082384 May 24 10:30:42.185096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 724/1374] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:42 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:43.202156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:43.202760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:43.209333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:43.209779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:43.210530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:43.210960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:43.271178 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:43.279296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8ab167eb-04f0-45f2-9428-876274ec6db2 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081201 May 24 10:30:43.279851 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 651/1375] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:43 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:44.297204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:44.297858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:44.304642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:44.305094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:44.305892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:44.306331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:44.373876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:44.381598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3da72b1c-667b-4489-8f40-9147f3ba38d9 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.088231 May 24 10:30:44.382204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 725/1376] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:44 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:30:45.398739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:45.399334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:45.406439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:45.406878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:45.407639 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:45.408065 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:45.472593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:45.480499 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-07139755-17b2-49a2-a931-a736ebc6788e tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.085618 May 24 10:30:45.481071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 652/1377] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:45 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:46.497401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:46.498014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:46.504441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:46.504885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:46.505674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:46.506107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:46.570034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:46.577622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-93b5017a-477b-4e62-81a0-c28876653f50 tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.084183 May 24 10:30:46.578226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 726/1378] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:46 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:47.594397 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:47.595009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] [instance: 426e8866-95a1-42e7-9e88-29f93c5e7d7b] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:47.601613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:47.602063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:47.602823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:47.603260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:47.663828 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '577c7840904d4f2180f7e5bd0f530a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bf4c198860648cd874233ba27d1b971', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:47.671888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8c72edcf-6720-48e7-9ea1-ba503f78c1fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b" status: 200 len: 1210 microversion: 2.1 time: 0.081313 May 24 10:30:47.672449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 653/1379] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:47 2022] GET /compute/v2.1/servers/426e8866-95a1-42e7-9e88-29f93c5e7d7b => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:30:47.897894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c02cc134-0b55-4558-8a86-3737fb7a31fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] Calling method '.version_select at 0x7f8737555550>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:47.991425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c02cc134-0b55-4558-8a86-3737fb7a31fe tempest-TestShelveInstance-99604636 tempest-TestShelveInstance-99604636-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-91633904" status: 202 len: 0 microversion: 2.1 time: 0.097341 May 24 10:30:47.992007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 727/1380] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:30:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-91633904 => generated 0 bytes in 99 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:30:58.430712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:58.431236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:58.433792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=17 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:58.434615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=71 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:58.443429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:58.443429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:58.443429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:58.443429 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:58.471225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Going to try to terminate instance {{(pid=118233) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:30:58.880300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c1f10aa0-4fba-49fc-947a-31896c81a619 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 204 len: 0 microversion: 2.1 time: 0.453105 May 24 10:30:58.880976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 654/1381] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:30:58 2022] DELETE /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:30:58.888377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=47 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:58.888377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=20,nova_api:DELETE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:58.891740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:30:58.892382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:30:58.899149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:58.899630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:58.900396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:30:58.901650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:30:59.071585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:30:59.080817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-66178c7d-dd26-46a4-99e6-936ca43012aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.195806 May 24 10:30:59.083162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 728/1382] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:30:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:00.099641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:00.102681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:00.107969 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:00.108426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:00.109196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:00.109661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:00.247105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:00.255166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f2c64e3a-2528-468d-b810-49c26f0ca449 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1734 microversion: 2.1 time: 0.160114 May 24 10:31:00.255676 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 655/1383] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:00.857489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-7e49c5a6-a3d8-4358-994b-20d15cb2da72 req-b8742b68-45b1-4889-98e0-82d83868c1b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "79059316-a25c-4dd7-954c-0f7a413203a1", "name": "network-vif-deleted", "tag": "6c9affb3-0667-4743-a73b-88352bae25e5"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:31:00.866157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7e49c5a6-a3d8-4358-994b-20d15cb2da72 req-b8742b68-45b1-4889-98e0-82d83868c1b1 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:00.866862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7e49c5a6-a3d8-4358-994b-20d15cb2da72 req-b8742b68-45b1-4889-98e0-82d83868c1b1 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:00.881726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-7e49c5a6-a3d8-4358-994b-20d15cb2da72 req-b8742b68-45b1-4889-98e0-82d83868c1b1 service nova] Creating event network-vif-deleted:6c9affb3-0667-4743-a73b-88352bae25e5 for instance 79059316-a25c-4dd7-954c-0f7a413203a1 on hv-cinder-84311 May 24 10:31:00.892512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-7e49c5a6-a3d8-4358-994b-20d15cb2da72 req-b8742b68-45b1-4889-98e0-82d83868c1b1 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.222848 May 24 10:31:00.894218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 729/1384] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:31:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:31:00.966289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:00.966799 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:00.989134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:00.989617 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:00.990588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:00.992426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:01.025159 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Going to try to terminate instance {{(pid=118233) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:31:01.274371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:01.275291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:01.281900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:01.282273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:01.283039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:01.283469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:01.353141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:01.371582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-18f1d164-24c2-4eb2-834f-b6b2f3e50f7c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.102085 May 24 10:31:01.372205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 730/1385] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:01.459589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7ffbff92-f015-4662-a2cc-99b59c92d1ef tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "DELETE /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 204 len: 0 microversion: 2.1 time: 0.496942 May 24 10:31:01.460788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 656/1386] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:31:00 2022] DELETE /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:31:01.470117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:01.470862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:01.477346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:01.477904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:01.478714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:01.479287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:01.630197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:01.638972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-247d200e-0026-4a4f-86c1-f437c96b7c98 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1554 microversion: 2.1 time: 0.172889 May 24 10:31:01.639771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 731/1387] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:01 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1554 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:31:02.389145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:02.389886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:02.396845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:02.397327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:02.398164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:02.398610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:02.470637 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:02.479574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-067c2f1b-0139-4fb6-9a28-2fa39359b5ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.095144 May 24 10:31:02.480366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 657/1388] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:31:02.656062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:02.657804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:02.665168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:02.665848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:02.667521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:02.668251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:02.773130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:02.782566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3f193cb7-bcb2-475d-8476-166083cabfcc tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1512 microversion: 2.1 time: 0.130239 May 24 10:31:02.783169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 732/1389] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:02 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1512 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:02.923723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [req-12f9b08f-f318-43e2-863d-4a39b11b9619 req-3cc2434a-9545-4c33-9a98-f0c7f5cfe564 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6242a514-2829-4fa8-8572-96bca164b6a1", "name": "network-vif-deleted", "tag": "a72bde82-4d88-4a59-8195-0159fccc9ac0"}]} {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:31:02.932394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-12f9b08f-f318-43e2-863d-4a39b11b9619 req-3cc2434a-9545-4c33-9a98-f0c7f5cfe564 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:02.933117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [req-12f9b08f-f318-43e2-863d-4a39b11b9619 req-3cc2434a-9545-4c33-9a98-f0c7f5cfe564 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:02.948237 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.compute.server_external_events [req-12f9b08f-f318-43e2-863d-4a39b11b9619 req-3cc2434a-9545-4c33-9a98-f0c7f5cfe564 service nova] Creating event network-vif-deleted:a72bde82-4d88-4a59-8195-0159fccc9ac0 for instance 6242a514-2829-4fa8-8572-96bca164b6a1 on hv-cinder-84311 May 24 10:31:02.955113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [req-12f9b08f-f318-43e2-863d-4a39b11b9619 req-3cc2434a-9545-4c33-9a98-f0c7f5cfe564 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034815 May 24 10:31:02.956010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 658/1390] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:31:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:31:03.498448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:03.500965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:03.511068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:03.511964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:03.513488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:03.514348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:03.656858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:03.667109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7fba0424-3557-4147-b4bd-564c40967141 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.174937 May 24 10:31:03.668782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 733/1391] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:03.801704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:03.802507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:03.810141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:03.810906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:03.812051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:03.812607 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:03.849611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:03.850153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:03.850699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.objects.instance [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lazy-loading 'fault' on Instance uuid 6242a514-2829-4fa8-8572-96bca164b6a1 {{(pid=118233) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:31:03.894403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77f178a1600c4704a9f7dba7e3b8f36a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2606d8cef9c442c896c12b401c92326f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:03.903107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-addd6814-f0e5-4313-a235-fdbe2335b18a tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 200 len: 1348 microversion: 2.1 time: 0.106193 May 24 10:31:03.908533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 659/1392] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:03 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 1348 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:04.694710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:04.695360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:04.703748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:04.704255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:04.704979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:04.705636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:04.812154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:04.821321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f86c7ef5-0daf-46e7-b976-e4e9251c45c9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.130597 May 24 10:31:04.822471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 734/1393] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:04.920342 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:04.921542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] [instance: 6242a514-2829-4fa8-8572-96bca164b6a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:04.929355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:04.930495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:04.935947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:04.935947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:04.950532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] HTTP exception thrown: Instance 6242a514-2829-4fa8-8572-96bca164b6a1 could not be found. May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Request method failure captured: May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 HTTP/1.1 May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijLB2cL3VamoKw7gkh7F7K2wtR1aaAGBnMSYbPPC3gA1zkw8trVbuRR0V4sTPYa1ilQhTbQv-PV4_PYIP1K0Xhx0yN485CfvJSTH7S3rLcOpeAJgBhlB9zE6RoG0MSHEArotC3QcNTUMSpyHhC1YSx4hiumbeV4A9iNgdOGUy-MepWHs May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: 2606d8cef9c442c896c12b401c92326f May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-TestMinimumBasicScenario-694433787 May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:31:04.952267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/2606d8cef9c442c896c12b401c92326f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_2606d8cef9c442c896c12b401c92326f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/2606d8cef9c442c896c12b401c92326f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/2606d8cef9c442c896c12b401c92326f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-TestMinimumBasicScenario-694433787 May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: 2606d8cef9c442c896c12b401c92326f May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-TestMinimumBasicScenario-694433787 May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-TestMinimumBasicScenario-694433787-project May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 77f178a1600c4704a9f7dba7e3b8f36a May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-TestMinimumBasicScenario-694433787-project May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Instance 6242a514-2829-4fa8-8572-96bca164b6a1 could not be found. May 24 10:31:04.953944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': '6242a514-2829-4fa8-8572-96bca164b6a1'} May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6242a514-2829-4fa8-8572-96bca164b6a1 could not be found. May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:31:04.954856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6242a514-2829-4fa8-8572-96bca164b6a1 could not be found. May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6242a514-2829-4fa8-8572-96bca164b6a1 could not be found. May 24 10:31:04.956156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:31:04.957332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Returning 404 to user: Instance 6242a514-2829-4fa8-8572-96bca164b6a1 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:31:04.957332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-73bd680c-1c66-4c61-91b0-21307cbfa5ab tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1" status: 404 len: 111 microversion: 2.1 time: 0.037580 May 24 10:31:04.957332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 660/1394] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:04 2022] GET /compute/v2.1/servers/6242a514-2829-4fa8-8572-96bca164b6a1 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:31:04.962309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8501e90a-c248-4d9c-ad40-b8ccafed51a0 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] Calling method '.version_select at 0x7f87375864c0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:05.044177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8501e90a-c248-4d9c-ad40-b8ccafed51a0 tempest-TestMinimumBasicScenario-694433787 tempest-TestMinimumBasicScenario-694433787-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-439592202" status: 202 len: 0 microversion: 2.1 time: 0.085492 May 24 10:31:05.044785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 735/1395] 10.222.0.35 () {58 vars in 1268 bytes} [Tue May 24 10:31:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-439592202 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:31:05.839586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:05.840542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:05.847419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:05.847901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:05.849500 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:05.850188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:05.925293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:05.936122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cc43de81-a355-4585-9c57-4f9e5280980e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.100779 May 24 10:31:05.937577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 661/1396] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:06.955123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:06.955838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:06.966623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:06.969634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:06.971533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:06.972004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:07.079621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:07.097575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-764cdfe8-f73a-4b2c-9466-c74cb5ebebd7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.146380 May 24 10:31:07.101295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 736/1397] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:08.115111 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:08.116448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:08.124807 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:08.132768 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:08.132768 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:08.132768 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:08.219626 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:08.230783 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5c92ef16-d5f5-40f8-8efa-7c5b5753d1b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.119615 May 24 10:31:08.239879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 662/1398] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:09.247808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:09.249456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:09.262705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:09.263159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:09.269881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:09.270667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:09.371904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:09.384534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-145459e5-b504-411b-8060-cb2a5dcec82c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.140512 May 24 10:31:09.385669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 737/1399] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:10.418899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:10.422972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:10.435508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:10.435956 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:10.437048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:10.437528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:10.571414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:10.581390 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0190f59e-bf01-41da-ae52-93db17f6ee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.177994 May 24 10:31:10.592746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 663/1400] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:11.601065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:11.603025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:11.609811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:11.616743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:11.626443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:11.626443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:11.731777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:11.742313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5764b4b1-e3b7-412f-b4aa-af3046717f7a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.146305 May 24 10:31:11.743033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 738/1401] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:12.761749 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:12.762530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:12.771434 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:12.773904 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:12.775406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:12.777687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:12.870802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:12.879528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-faacbf59-3250-41be-9b74-3c54dd0ca39a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.123031 May 24 10:31:12.880691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 664/1402] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:13.897570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:13.898186 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:13.909448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:13.910127 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:13.911075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:13.911855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:13.925256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-92663502-6930-4a89-b627-a987c52fbb30 tempest-TestVolumeBootPattern-1466413927 tempest-TestVolumeBootPattern-1466413927-project] Action: 'create', calling method: .version_select at 0x7f87374f7310>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-801497010", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBByYZSo7ale6ehMKwvhhRMG//6+q0CYEaRZerQ/ygRWgT3W7rzImnL5eqFBOZg82Tcl/A6gq9PXL24B2293jIaukPVvmJereA5nDCiRAV9ogF4sCVTkn6NXr8Te6Y63TUQ=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:31:13.943890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-92663502-6930-4a89-b627-a987c52fbb30 tempest-TestVolumeBootPattern-1466413927 tempest-TestVolumeBootPattern-1466413927-project] Getting quotas for project 31a105514cfa456d9de3a0b5903b2568. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:31:13.948974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-92663502-6930-4a89-b627-a987c52fbb30 tempest-TestVolumeBootPattern-1466413927 tempest-TestVolumeBootPattern-1466413927-project] Getting quotas for user 5a12e3c1ed3f41b699a5379d120396da and project 31a105514cfa456d9de3a0b5903b2568. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:31:14.012577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:14.020986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cd8a9783-c30f-4b88-a97a-db437b16d49e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.127360 May 24 10:31:14.030225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 665/1403] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:14.096584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-92663502-6930-4a89-b627-a987c52fbb30 tempest-TestVolumeBootPattern-1466413927 tempest-TestVolumeBootPattern-1466413927-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.350585 May 24 10:31:14.102680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 739/1404] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:31:13 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:31:15.038714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:15.040565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:15.059898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:15.060458 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:15.061406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:15.062187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:15.149548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:15.161518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d781759f-3951-4d8f-a618-3d5bfc5c4828 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.126887 May 24 10:31:15.171110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 666/1405] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:16.179447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:16.180110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:16.189515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:16.190301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:16.191253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:16.191849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:16.284410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:16.293344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e2de0b6e-4c6f-418a-8a77-f089aecb824b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.118610 May 24 10:31:16.293930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 740/1406] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:17.312861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:17.312861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:17.331664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:17.332426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:17.333675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:17.334274 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:17.419183 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:17.427944 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-43943ce4-9f36-4632-8f4b-f72d3cb15d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.121719 May 24 10:31:17.428611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 667/1407] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:18.446031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:18.446646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:18.453429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:18.453874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:18.454702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:18.455134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:18.533892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:18.543326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e377c5e8-fbca-431a-a7b1-849578362b3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.101552 May 24 10:31:18.543636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 741/1408] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:19.559804 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:19.560414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:19.567155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:19.567668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:19.568420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:19.568853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:19.572991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:19.630306 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:19.638553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-40e9cf65-517c-4b9b-9347-dccf77fd3cb1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082735 May 24 10:31:19.639147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 668/1409] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:20.655855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:20.656495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:20.663140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:20.663589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:20.664356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:20.664782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:20.726834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:20.734986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6ec2f935-08f5-46d4-8e77-a5ebdf389fa0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083030 May 24 10:31:20.735564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 742/1410] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:21.751752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:21.752364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:21.760113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:21.760565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:21.761355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:21.761795 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:21.828832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:21.837142 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f69c7c7d-618e-44ac-916d-44c07454b9de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.089210 May 24 10:31:21.837765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 669/1411] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:22.854278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:22.854981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:22.861327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:22.861793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:22.862551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:22.862986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:22.923353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:22.931269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e04719d6-27df-40c1-b7fd-2b15d7662dad tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081497 May 24 10:31:22.931846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 743/1412] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:23.949660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:23.950278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:23.956722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:23.957174 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:23.957978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:23.958419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:24.018292 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:24.026448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-69b0f1a3-5419-4938-bfeb-4e4265a19b6f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081553 May 24 10:31:24.027006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 670/1413] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:25.043347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:25.043965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:25.046378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:25.051258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:25.051704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:25.052485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:25.052903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:25.116471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:25.124520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-37de2574-6a67-48c3-86e2-a5b273f589e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085501 May 24 10:31:25.125100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 744/1414] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:26.141890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:26.142508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:26.149199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:26.149796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:26.150563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:26.151512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:26.213063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:26.221818 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e1fb721a-7cef-4479-bbbc-983b372f4f59 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083801 May 24 10:31:26.222387 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 671/1415] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:27.238324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:27.238933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:27.246243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:27.246697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:27.247461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:27.247904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:27.308202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:27.316377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bc445b3c-92be-42a6-b80a-be106c1f0ab9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081966 May 24 10:31:27.316942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 745/1416] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:28.333046 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:28.333679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:28.340377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:28.340822 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:28.341614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:28.342055 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:28.402527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:28.410561 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bd54dfd0-4e6c-4369-b0a2-453e70c4209e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081388 May 24 10:31:28.411189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 672/1417] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:29.428304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:29.428951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:29.436589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:29.437409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:29.438186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:29.438622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:29.442715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=93 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:29.501199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:29.509334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a6e27e31-e487-4f19-8f66-745577424c16 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085146 May 24 10:31:29.509930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 746/1418] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:30.526092 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:30.526875 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:30.529199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=2 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:30.533989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:30.534431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:30.535185 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:30.535615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:30.606715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:30.615098 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e5134a63-f665-4b3c-9f61-28b375af4b6a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.092892 May 24 10:31:30.615656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 673/1419] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:31.632932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:31.633655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:31.636695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=49,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:31.641657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:31.642111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:31.642878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:31.643311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:31.706342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:31.714839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f87da22e-49e7-44a5-b594-79fa8a59d364 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086313 May 24 10:31:31.715410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 747/1420] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:32.732370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:32.732981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:32.739730 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:32.740181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:32.740954 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:32.741449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:32.804096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:32.812491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4d6f5697-ccdb-48a6-8de9-5305019d35f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084700 May 24 10:31:32.813060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 674/1421] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:33.830010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:33.830611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:33.837353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:33.837809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:33.838778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:33.839345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:34.067465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:34.075779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ac0bf414-dccb-492a-bd4c-808510d8fdf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.249996 May 24 10:31:34.076567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 748/1422] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:35.091826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:35.092439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:35.098949 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:35.099395 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:35.100164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:35.100599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:35.162034 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:35.170108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b0b4ff77-2c97-475d-9f32-f8f03594acec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082159 May 24 10:31:35.170667 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 675/1423] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:36.196980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:36.197616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:36.204810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:36.205257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:36.206034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:36.206466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:36.266590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:36.274338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b87807cd-42c7-49f9-b893-ca8106a74bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081238 May 24 10:31:36.274897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 749/1424] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:37.293101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:37.293733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:37.300281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:37.300660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:37.301449 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:37.301915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:37.365124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:37.373139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7b6ef261-fa63-4839-890b-95df973ad86a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084638 May 24 10:31:37.373732 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 676/1425] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:38.390040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:38.390677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:38.397140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:38.397620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:38.398382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:38.398818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:38.458965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:38.466933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7257e6c0-32e2-421a-ba2a-b6b02e074468 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081373 May 24 10:31:38.467487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 750/1426] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:39.484411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:39.485027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:39.491579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:39.492026 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:39.492784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:39.493220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:39.557987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:39.566406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bd43f07a-3e74-4a83-a3db-07582e245940 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085892 May 24 10:31:39.566972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 677/1427] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:40.583953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:40.584609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:40.591357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:40.591801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:40.592561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:40.592990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:40.653997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:40.661793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-44e09bb3-0b94-434e-a097-237908c9c839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082674 May 24 10:31:40.662347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 751/1428] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:41.678645 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:41.679260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:41.687198 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:41.687940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:41.688722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:41.689161 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:41.752487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:41.761486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b632da9-10dd-492e-8bb7-49efc868aca9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086642 May 24 10:31:41.762158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 678/1429] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:42.778507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:42.779114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:42.785658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:42.786106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:42.786868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:42.787302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:42.853955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:42.862438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-709fa1d0-22e6-422e-8f12-d44d3b5959e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088465 May 24 10:31:42.862996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 752/1430] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:43.879333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:43.879886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:43.886742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:43.887198 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:43.887967 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:43.888430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:43.951132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:43.959222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-93fd9ccf-23bd-416f-acde-329bed07495d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083889 May 24 10:31:43.959789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 679/1431] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:44.976788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:44.977499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:44.984079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:44.984519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:44.985290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:44.985727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:45.045663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:45.053604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1e083a98-99de-4f9b-92f3-a14e96696b3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080654 May 24 10:31:45.054169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 753/1432] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:46.070980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:46.071619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:46.078313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:46.078761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:46.079522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:46.079961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:46.143857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:46.150184 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ccca940b-d69c-4ec4-b158-795e5d92133d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083036 May 24 10:31:46.150695 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 680/1433] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:47.167478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:47.168090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:47.174820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:47.177961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:47.177961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:47.177961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:47.241758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:47.251165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9da9a630-9414-4d9f-95d5-8afbb0153479 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087408 May 24 10:31:47.251728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 754/1434] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:48.268173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:48.268777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:48.275357 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:48.276462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:48.277834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:48.278268 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:48.342725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:48.350550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4eef8f13-a247-4ece-87dc-2d5801c54b54 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086238 May 24 10:31:48.351113 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 681/1435] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:49.367275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:49.367833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:49.374240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:49.374684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:49.375437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:49.375865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:49.437347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:49.445593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2fe9adbf-a7a6-4af4-8b07-31617ba31013 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082302 May 24 10:31:49.446166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 755/1436] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:50.462303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:50.462918 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:50.469624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:50.470071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:50.470864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:50.471303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:50.533533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:50.542062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-13fa452b-7500-49b8-b3b8-f6c9b391bfec tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083108 May 24 10:31:50.542635 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 682/1437] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:51.559299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:51.559901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:51.566648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:51.567094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:51.567846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:51.568278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:51.629243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:51.638063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-198c77d3-61a8-4c0f-8db5-caac41685933 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083154 May 24 10:31:51.638624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 756/1438] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:52.655142 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:52.655750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:52.662143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:52.662588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:52.663347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:52.663779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:52.667653 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=89 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:52.726267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:52.734191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a96b5eb3-8316-4183-b4a6-29db71b7594c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083612 May 24 10:31:52.734758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 683/1439] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:53.751046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:53.751679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:53.758435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:53.758890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:53.759664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:53.760102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:53.824191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:53.832140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a363c3d0-4ce4-489e-a517-d3a2fb44c317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085018 May 24 10:31:53.832695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 757/1440] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:31:54.849465 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:54.850071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:54.857255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:54.857724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:54.858481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:54.858911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:54.921864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:54.929892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d48ab211-b8f5-4545-ab92-5b6f43d432c4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084382 May 24 10:31:54.930461 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 684/1441] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:55.947830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:55.948423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:55.956685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:55.957153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:55.957955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:55.958382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:56.019562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:56.027580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-09cab2ce-95c6-4f85-8472-8cfbb877c373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083526 May 24 10:31:56.028139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 758/1442] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:57.044896 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:57.045540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:57.052036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:57.052485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:57.053247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:57.053712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:57.118596 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:57.126696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-48c949be-90c9-4419-965d-f4393cc8119c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086445 May 24 10:31:57.127265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 685/1443] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:58.143273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:58.143884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:58.150384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:58.150829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:58.151593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:58.152080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:58.213051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:58.221729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ed451b1c-f2d4-4db3-b0be-0e27963380f4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082232 May 24 10:31:58.222295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 759/1444] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:31:59.238471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:31:59.239073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:31:59.246982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:59.247420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:59.248204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:31:59.248636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:31:59.310708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:31:59.318816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-45f0fcbd-bcc8-47f4-bd45-90ac829477fd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084189 May 24 10:31:59.319380 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 686/1445] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:31:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:00.247900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:32:00.248758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:32:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:32:00.335961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:00.336654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:00.343353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:00.343800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:00.344563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:00.344998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:00.406984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:00.414969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0530d61-4501-4eb8-bca5-38da287d50f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082900 May 24 10:32:00.415475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 760/1446] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:01.432044 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:01.432609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:01.439940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:01.440393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:01.441162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:01.441629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:01.502378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:01.510269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0881ef37-86d7-4b52-82a4-7bf6489c85e8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082226 May 24 10:32:01.510826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 687/1447] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:02.527548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:02.528190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:02.535258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:02.535700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:02.536453 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:02.536884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:02.539404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=90 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:02.602161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:02.610207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-30493df1-6f6a-4fbe-b49a-7e9666a0c12c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086537 May 24 10:32:02.610765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 761/1448] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:03.627261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:03.627870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:03.629903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=30 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:03.634737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:03.635213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:03.635976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:03.636403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:03.698569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:03.706644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cab2dbd5-cc9a-42f9-9937-b1ea8324e96d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083257 May 24 10:32:03.707213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 688/1449] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:04.723638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:04.724249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:04.726689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=30 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:04.731461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:04.731902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:04.733249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:04.734006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:04.795912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:04.804013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2c96f6cb-70df-4d0c-9e6d-e84875b1d85f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084262 May 24 10:32:04.804584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 762/1450] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:05.820753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:05.821413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:05.828897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:05.829388 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:05.830155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:05.830590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:05.896940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:05.904830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ab80fdf6-2975-4895-8a6c-5fb4e6e69a0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087976 May 24 10:32:05.905838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 689/1451] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:06.307319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:32:06.307863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:32:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:32:06.920475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:06.921078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:06.927664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:06.928107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:06.928899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:06.929351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:06.990434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:06.998154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ee085c19-b9c6-49b4-9072-f1fc874837ed tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081592 May 24 10:32:06.998890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 763/1452] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:08.015155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:08.015737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:08.022249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:08.022632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:08.023383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:08.023810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:08.083872 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:08.091781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-85cb6bce-24e0-41fe-8a8a-ba6e047e588f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080685 May 24 10:32:08.092852 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 690/1453] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:09.109667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:09.110407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:09.116590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:09.129510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:09.129510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:09.129510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:09.179607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:09.188725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f04f5b76-36e4-4dc2-b1b4-619b3efc7179 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083504 May 24 10:32:09.189304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 764/1454] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:10.206059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:10.206697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:10.213293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:10.213733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:10.214486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:10.214916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:10.275965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:10.283962 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-aeffd415-0673-4f2a-9e31-8a70306cc0bf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082562 May 24 10:32:10.284530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 691/1455] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:11.301126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:11.301763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:11.308235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:11.308682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:11.309497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:11.309935 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:11.373034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:11.381624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7c034078-331d-43f3-ace2-1d621e86f2d2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084379 May 24 10:32:11.382191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 765/1456] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:12.398779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:12.399380 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:12.406556 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:12.406993 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:12.407748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:12.408176 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:12.468753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:12.476612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-92d016dd-5d81-403f-91fc-137f1aedae0b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081685 May 24 10:32:12.477172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 692/1457] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:13.493292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:13.493903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:13.501123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:13.501595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:13.502363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:13.502805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:13.563954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:13.571907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a51d75cc-fb5d-4edb-abc4-0f2f81f4664b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082571 May 24 10:32:13.572476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 766/1458] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:14.588674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:14.589295 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:14.596551 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:14.597386 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:14.598269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:14.598715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:14.665031 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:14.672976 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fd6ae0e7-ff65-4baa-8b51-ddf37dee5d36 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088278 May 24 10:32:14.673574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 693/1459] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:15.691063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:15.691660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:15.698466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:15.698911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:15.699670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:15.700109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:15.763185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:15.771145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-984c8d72-5f75-4d3c-8423-0facbaa6c656 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084866 May 24 10:32:15.771701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 767/1460] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:16.788611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:16.789208 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:16.795810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:16.796255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:16.797016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:16.797491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:16.857734 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:16.865632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-18ce1f70-a1f4-4290-bb0e-c2297777b432 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080940 May 24 10:32:16.866200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 694/1461] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:17.418075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3fd0df08-5039-455d-b4f8-780540f40f80 tempest-TestVolumeBootPattern-1466413927 tempest-TestVolumeBootPattern-1466413927-project] Calling method '.version_select at 0x7f8732177d30>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:17.499262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3fd0df08-5039-455d-b4f8-780540f40f80 tempest-TestVolumeBootPattern-1466413927 tempest-TestVolumeBootPattern-1466413927-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-801497010" status: 202 len: 0 microversion: 2.1 time: 0.085156 May 24 10:32:17.499857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 768/1462] 10.222.0.35 () {58 vars in 1259 bytes} [Tue May 24 10:32:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-801497010 => generated 0 bytes in 86 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:32:17.882293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:17.882908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:17.889330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:17.889787 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:17.890577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:17.890994 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:17.953195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:17.961698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-464f05e8-52d0-4292-83dd-708a4d8b38f7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083330 May 24 10:32:17.962272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 695/1463] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:18.978416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:18.979629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:18.987782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:18.988224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:18.988997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:18.989454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:19.062939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:19.088227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-009c8c24-b1df-4d0b-bde2-cc191e7c2289 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.113708 May 24 10:32:19.089155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 769/1464] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:20.105967 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:20.106588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:20.113392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:20.113845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:20.114618 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:20.115053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:20.181947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:20.190488 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-55114720-0f55-4f4f-b353-61f726abb901 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088283 May 24 10:32:20.191134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 696/1465] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:21.212067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:21.212067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:21.215603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:21.216058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:21.216823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:21.217256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:21.280882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:21.289784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e260b244-ccfc-4a2d-a6e5-0449e25392c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085716 May 24 10:32:21.299607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 770/1466] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:22.307134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:22.307736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:22.314722 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:22.315169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:22.316503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:22.316942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:22.400356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:22.408329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8462d0ef-689a-4338-9d80-27f7a7dc2348 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.105007 May 24 10:32:22.409112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 697/1467] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:23.426089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:23.426871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:23.434575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:23.435020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:23.435790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:23.436519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:23.500588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:23.509159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-23af309a-ccb3-47e3-858c-049e72f5d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087639 May 24 10:32:23.509687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 771/1468] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:24.526345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:24.527091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:24.534418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:24.534996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:24.535901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:24.536435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:24.539403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=90 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:24.608424 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:24.620074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-71116ddf-703c-4704-8e11-f357a5463eb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.096950 May 24 10:32:24.621405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 698/1469] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:25.637980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:25.650218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:25.650218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:25.650218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:25.650218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:25.650634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:25.712462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:25.720680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fff40ec1-6fdc-4b60-bafe-1bea3ec98cbc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086556 May 24 10:32:25.721244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 772/1470] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:26.736790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:26.737597 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:26.744293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:26.744742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:26.745555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:26.745986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:26.808757 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:26.817612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dc8ba2ac-51a6-456c-ae42-d0e112750c3f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084255 May 24 10:32:26.820773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 699/1471] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:27.835444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:27.836066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:27.842551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:27.843000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:27.843770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:27.844208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:27.905372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:27.913974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-acd17641-f537-43ac-be60-ec5be93b67e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083358 May 24 10:32:27.914544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 773/1472] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:28.931696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:28.932246 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:28.938322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:28.939022 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:28.939777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:28.940204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:29.001102 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:29.009181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-71ab7ff5-4887-4722-91a6-a209d168b87f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082680 May 24 10:32:29.009771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 700/1473] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:30.027565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:30.028175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:30.034674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:30.035125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:30.035886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:30.036321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:30.101860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:30.109665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9b0fcd3a-7e59-4a26-824d-9f7f422bb627 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085942 May 24 10:32:30.110240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 774/1474] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:31.125812 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:31.126422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:31.133718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:31.134162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:31.134919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:31.135351 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:31.200041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:31.204974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-58911243-5eab-4270-bca7-425c6d424848 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082991 May 24 10:32:31.205562 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 701/1475] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:32.222502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:32.223120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:32.230457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:32.230906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:32.231661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:32.232092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:32.294060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:32.302099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-70559707-5e38-4824-9f53-4fba2253e068 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083500 May 24 10:32:32.302665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 775/1476] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:33.319253 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:33.319878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:33.326260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:33.326651 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:33.327411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:33.327842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:33.393935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:33.401930 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-97b4bcae-f047-4e07-9223-f52f48fa98c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086545 May 24 10:32:33.402495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 702/1477] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:34.418798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:34.419346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:34.425851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:34.426322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:34.427048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:34.427480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:34.431310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=84 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:34.488699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:34.497195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-00b0afaf-6a8b-4cbe-bc5b-fa2d00f95f08 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082430 May 24 10:32:34.497815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 776/1478] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:35.514609 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:35.515214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:35.517507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=30 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:35.522398 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:35.522847 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:35.523612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:35.524049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:35.586611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:35.595363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9b3b97a3-bb13-4928-90ae-f7216d091a9a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085331 May 24 10:32:35.595935 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 703/1479] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:36.612659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:36.613286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:36.615765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=31,nova_api:DELETE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:36.620602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:36.621052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:36.621861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:36.622290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:36.683783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:36.692510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8cb5b16a-ce10-451e-bebb-de44618baf5f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083758 May 24 10:32:36.693076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 777/1480] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:37.709192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:37.709831 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:37.717051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:37.717524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:37.718284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:37.718715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:37.779670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:37.787508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c61e87f6-a66c-4a11-9ad2-042928b0a928 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082205 May 24 10:32:37.788071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 704/1481] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:38.803992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:38.804606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:38.811494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:38.811931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:38.812699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:38.813134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:38.876989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:38.884937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ea765f24-6a0d-4bf3-a23e-5ffecd0b037a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085083 May 24 10:32:38.885737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 778/1482] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:39.902525 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:39.903724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:39.910207 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:39.910665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:39.911423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:39.911858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:39.973545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:39.981520 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ffa18f52-3b40-4193-8795-0718d648f259 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082876 May 24 10:32:39.982086 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 705/1483] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:40.999074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:40.999679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:41.006208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:41.006656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:41.007417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:41.007852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:41.068848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:41.076819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7650d321-c582-4c56-be2d-b3df314d42ff tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081921 May 24 10:32:41.077409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 779/1484] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:42.092586 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:42.093191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:42.099677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:42.100121 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:42.100876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:42.101339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:42.165392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:42.175381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f0164883-2d5c-461b-be88-c1c9635989ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086727 May 24 10:32:42.176048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 706/1485] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:43.193238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:43.193875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:43.201169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:43.201652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:43.202498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:43.202942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:43.264699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:43.272580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b3ff7de2-bccc-4684-a67f-ac7e2d4e2d43 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083514 May 24 10:32:43.273145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 780/1486] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:44.289890 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:44.290604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:44.297698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:44.298147 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:44.298911 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:44.299342 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:44.364494 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:44.372447 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bb3d4f1d-cd39-480d-8080-0b96205ecb89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086371 May 24 10:32:44.373003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 707/1487] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:45.389061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:45.389636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:45.396238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:45.396691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:45.397498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:45.397937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:45.461390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:45.470182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8feccac1-e4aa-4e66-845b-094330c4e060 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085452 May 24 10:32:45.470754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 781/1488] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:46.487516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:46.488099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:46.494752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:46.495196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:46.495946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:46.496376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:46.556977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:46.565717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d7d54241-3f2a-4c6b-a8c5-34ca510d8e01 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082133 May 24 10:32:46.566281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 708/1489] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:47.582170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:47.582790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:47.589435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:47.589868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:47.590626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:47.591110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:47.655542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:47.667493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a118eac-1961-41c4-b270-4ec4bab292e0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088887 May 24 10:32:47.668080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 782/1490] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:48.684374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:48.685408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:48.692462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:48.692915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:48.693714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:48.694155 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:48.757871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:48.766532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-17c078a7-7f06-46af-a02a-768d457bc7ca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086008 May 24 10:32:48.767363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 709/1491] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:49.783575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:49.784159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:49.790635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:49.791086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:49.791846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:49.792278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:49.858015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:49.866005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1a0ff045-bc7e-4cb2-ae50-c8d35415a21b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086276 May 24 10:32:49.866573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 783/1492] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:50.882548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:50.883132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:50.889752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:50.890195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:50.890948 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:50.891376 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:50.957085 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:50.964996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ff2755f6-d476-4413-9057-72d40f8db691 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086343 May 24 10:32:50.965594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 710/1493] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:32:51.983063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:51.983689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:51.990236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:51.990687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:51.991438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:51.991869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:52.052054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:52.059902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2ed92f08-3473-4ee5-a2c3-5159f7f5ce25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081333 May 24 10:32:52.060554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 784/1494] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:53.075990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:53.076601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:53.083315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:53.083763 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:53.084528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:53.084971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:53.147063 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:53.155040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b0c9ac4-f72f-406d-be5d-0b2eb5143612 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082955 May 24 10:32:53.155721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 711/1495] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:54.171843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:54.172483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:54.179130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:54.179580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:54.180343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:54.180779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:54.243977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:54.252904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dbfe5a39-de92-4ab6-ab1c-3ea8670c0073 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085095 May 24 10:32:54.253505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 785/1496] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:55.270776 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:55.271391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:55.277908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:55.278336 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:55.279104 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:55.279538 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:55.282790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=84 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:55.345547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:55.353691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c3a87d16-a233-4c27-bb84-b6d9bb7b8733 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086804 May 24 10:32:55.354257 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 712/1497] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:56.370956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:56.371566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:56.378759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:56.379221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:56.380015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:56.380451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:56.441131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:56.449079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-33f41170-2a2f-49a2-b435-1fbfaa17d05d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082063 May 24 10:32:56.449672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 786/1498] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:57.466661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:57.467272 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:57.474355 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:57.474803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:57.475555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:57.475987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:57.538234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:57.546419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-90aaaa7f-097e-4f10-99b4-87ee399045a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083351 May 24 10:32:57.547004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 713/1499] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:32:58.563658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:58.564270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:58.571275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:58.571723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:58.572490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:58.572926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:58.636423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:58.644415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1a3c325b-865a-4d6b-9a21-fec05705900e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084889 May 24 10:32:58.645076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 787/1500] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:32:59.662167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:32:59.662688 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:32:59.669348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:59.669792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:59.670545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:32:59.671538 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:32:59.735361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:32:59.743199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3a978dbc-4bd8-45fb-9c99-0838a0907baa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085140 May 24 10:32:59.743761 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 714/1501] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:32:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:00.760372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:00.760976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:00.768086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:00.768529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:00.769308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:00.769740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:00.832338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:00.840175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9d02e177-5462-4f96-a015-a61c0d37be02 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083731 May 24 10:33:00.840736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 788/1502] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:01.856923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:01.857555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:01.864706 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:01.865149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:01.865939 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:01.866371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:01.930139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:01.938873 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-04cc7549-0597-443d-8d40-b5ee59b6850b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085832 May 24 10:33:01.939898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 715/1503] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:02.023579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:02.024093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:02.046236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:02.046236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:02.046236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:02.046236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:02.049999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] HTTP exception thrown: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Request method failure captured: May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 HTTP/1.1 May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_TJiJC50rIjdcS63u7xuYhM4UlnZwxRv8mUKp0BzcDusYujaMOjfiQ7B-7czQB8mmEz41Y8c7oBuVaj_T84-448VkgY0Pf5gNrx-P1niD879m3w66ryHsb9xDN21s3gpWLUkY4irhyPqPHyYMccDT3Xqb0fUHM9qSZRWOI7k2kkeY May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:33:02.051758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeTestJSON-1912151979 May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 7bffd469507b4dbc8d3146f051c8c8ad May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.053456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'd6651fff-37e6-4e58-9f84-5e58f2579414'} May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:33:02.054257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.055611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:33:02.056855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Returning 404 to user: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:33:02.056855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-acca5b60-8daf-4227-b2d1-f9e954c3fb0d tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 404 len: 111 microversion: 2.1 time: 0.034315 May 24 10:33:02.056855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 789/1504] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:33:02 2022] DELETE /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:33:02.063347 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:02.064211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: d6651fff-37e6-4e58-9f84-5e58f2579414] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:02.071374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:02.071374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:02.072118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:02.072666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:02.086680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.wsgi [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] HTTP exception thrown: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Request method failure captured: May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: request: GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 HTTP/1.1 May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept: application/json May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Accept-Encoding: identity May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Connection: close May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Length: 0 May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Content-Type: application/json May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Host: 10.222.0.35 May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Openstack-System-Scope: None May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: User-Agent: python-urllib3/1.26.9 May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Auth-Token: gAAAAABijK_TJiJC50rIjdcS63u7xuYhM4UlnZwxRv8mUKp0BzcDusYujaMOjfiQ7B-7czQB8mmEz41Y8c7oBuVaj_T84-448VkgY0Pf5gNrx-P1niD879m3w66ryHsb9xDN21s3gpWLUkY4irhyPqPHyYMccDT3Xqb0fUHM9qSZRWOI7k2kkeY May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Id: None May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Domain-Name: None May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Identity-Status: Confirmed May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Is-Admin-Project: True May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Id: default May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Domain-Name: Default May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Project-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Role: member,reader May 24 10:33:02.088255 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Roles: member,reader May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant: tempest-AttachVolumeTestJSON-1912151979 May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Id: default May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Domain-Name: Default May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Id: 7bffd469507b4dbc8d3146f051c8c8ad May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: X-User-Name: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: method: > May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: exception: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.089960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: action_args: {'id': 'd6651fff-37e6-4e58-9f84-5e58f2579414'} May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:33:02.090733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. May 24 10:33:02.091905 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: ERROR nova.api.openstack.wsgi May 24 10:33:02.093055 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Returning 404 to user: Instance d6651fff-37e6-4e58-9f84-5e58f2579414 could not be found. {{(pid=118233) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:33:02.093055 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-db192cdc-6c98-41dd-9b8a-fffa417ed61a tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414" status: 404 len: 111 microversion: 2.1 time: 0.031344 May 24 10:33:02.093055 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 716/1505] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:02 2022] GET /compute/v2.1/servers/d6651fff-37e6-4e58-9f84-5e58f2579414 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:33:02.956661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:02.957254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:02.963285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:02.963729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:02.964489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:02.964917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:03.025597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:03.034956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-19bac1da-a840-451d-b6b0-dfbff1edc284 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082848 May 24 10:33:03.035508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 790/1506] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:04.052700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:04.053660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:04.060410 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:04.060856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:04.062435 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:04.062681 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:04.125221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:04.133150 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e4f5a6c3-1f89-4e60-9fae-36ea020e643c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084411 May 24 10:33:04.133735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 717/1507] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:05.150186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:05.150792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:05.157627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:05.158072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:05.158840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:05.159271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:05.161857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=86 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:05.221717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:05.229657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-327f3aa4-04d4-48c9-a599-e807b971f916 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083264 May 24 10:33:05.230227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 791/1508] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:06.246860 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:06.247463 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:06.250536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=30 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:06.255339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:06.255832 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:06.256616 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:06.257047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:06.318955 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:06.326738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2ff1d86f-d8dd-4be4-8bfd-bb9157bb7868 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083801 May 24 10:33:06.328491 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 718/1509] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:07.345437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:07.346003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:07.348353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=30 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:07.353176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:07.353652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:07.354416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:07.354849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:07.418461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:07.426739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-34896a2d-2269-4049-85ec-535bd1b5b41e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086143 May 24 10:33:07.427307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 792/1510] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:08.443718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:08.444327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:08.451162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:08.451542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:08.452330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:08.452767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:08.520399 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:08.528585 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bce15a1f-37b7-49ad-950f-9587f4183454 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.089142 May 24 10:33:08.529146 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 719/1511] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:09.546893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:09.547500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:09.554135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:09.554586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:09.555357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:09.555794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:09.618489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:09.627494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1e1bfbc-7a45-4498-955a-cfda843aa032 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085132 May 24 10:33:09.628063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 793/1512] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:10.644179 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:10.644873 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:10.651433 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:10.651887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:10.652659 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:10.653100 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:10.714173 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:10.724312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-626f0fa1-1853-42a4-a63f-917d3d37b6d3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082931 May 24 10:33:10.724312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 720/1513] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:11.739932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:11.740565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:11.747061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:11.756241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:11.756241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:11.756241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:11.822062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:11.830735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-56454e00-6552-48da-ad13-4b6cefab11cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.094656 May 24 10:33:11.831573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 794/1514] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:12.847788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:12.848431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:12.855046 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:12.855516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:12.856287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:12.856727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:12.918441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:12.927195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-01672bc2-b31c-489e-82ea-f97c0f8831fb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083235 May 24 10:33:12.927765 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 721/1515] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:13.944432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:13.945037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:13.951800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:13.952416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:13.953178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:13.953637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:14.028994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:14.037501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c60ed77f-cc52-4286-8405-2697edadee3b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.097475 May 24 10:33:14.038079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 795/1516] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:15.055153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:15.055792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:15.063105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:15.063547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:15.064307 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:15.064737 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:15.128610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:15.136584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fea1a4bc-b0ce-4edf-9bc1-a5008bb395f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085306 May 24 10:33:15.137143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 722/1517] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:16.153767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:16.154353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:16.161216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:16.161691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:16.162458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:16.162887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:16.226001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:16.234421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-af818c49-dce5-4373-9770-2f67d065eb35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084580 May 24 10:33:16.234990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 796/1518] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:17.250223 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:17.250838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:17.257405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:17.257854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:17.258615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:17.259051 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:17.323760 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:17.332673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9a307b65-d141-4c48-a91d-cd45b1cccc68 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086367 May 24 10:33:17.333250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 723/1519] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:18.349665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:18.350292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:18.356681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:18.357126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:18.357914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:18.358598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:18.420431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:18.429125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-544d5009-64b5-4837-95d9-b429c4ae850f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083349 May 24 10:33:18.429731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 797/1520] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:19.446898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:19.447513 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:19.454607 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:19.455057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:19.455823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:19.456260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:19.517220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:19.525095 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5ee72a0a-cf1c-4256-9916-fd8852e97ca2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082029 May 24 10:33:19.525714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 724/1521] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:20.542482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:20.543092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:20.550164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:20.550620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:20.551419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:20.551858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:20.613064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:20.620947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d8ab5950-cd03-48d4-94dc-9bec2892b1a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082376 May 24 10:33:20.621563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 798/1522] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:21.639006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:21.640157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:21.646770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:21.647214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:21.647965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:21.648392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:21.710711 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:21.719442 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-50c138eb-feb4-46cd-aa68-1e2de4e64e9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084275 May 24 10:33:21.720010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 725/1523] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:22.736776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:22.737416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:22.743945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:22.744398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:22.745188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:22.745655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:22.806486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:22.814690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e759bee3-1f5b-468e-9821-c54b038f22f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082711 May 24 10:33:22.815513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 799/1524] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:23.831626 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:23.832377 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:23.838919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:23.839362 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:23.840117 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:23.840547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:23.901697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:23.909741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7c8a7ce5-eb19-431a-9b3d-60fcc02c209c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081991 May 24 10:33:23.910296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 726/1525] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:24.926483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:24.927091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:24.933603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:24.934052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:24.934806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:24.935238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:24.999868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:25.007975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8c4e8c7d-8d6f-4133-bc5d-2bb31eeebe58 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085371 May 24 10:33:25.008546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 800/1526] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:26.025633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:26.026271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:26.033215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:26.033685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:26.034441 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:26.034870 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:26.037423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=86 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:26.100180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:26.108089 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2fcc0f26-0b51-48fe-bbf3-dd82410e726a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086452 May 24 10:33:26.108652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 727/1527] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:27.123899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:27.124551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:27.131104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:27.131556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:27.132353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:27.132786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:27.196098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:27.204350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ba842c1f-5f31-4d1a-b12b-d7062ebed9a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084344 May 24 10:33:27.204916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 801/1528] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:28.222458 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:28.223072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:28.229970 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:28.230418 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:28.231168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:28.231598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:28.294383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:28.302999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-184e8194-5120-4e09-88b6-fa154ac6368f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085414 May 24 10:33:28.303566 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 728/1529] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:29.320391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:29.321003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:29.327746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:29.328192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:29.328957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:29.329424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:29.392720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:29.400361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-31cb4ca6-fafb-45c5-952b-962a4f16a07f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083993 May 24 10:33:29.400952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 802/1530] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:30.417420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:30.417937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:30.424878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:30.425350 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:30.426107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:30.426536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:30.487337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:30.495156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e597277a-3988-484e-ae39-5b7da750eaba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081739 May 24 10:33:30.495715 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 729/1531] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:31.512926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:31.513567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:31.520987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:31.521483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:31.522257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:31.522691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:31.597992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:31.605956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-52ce2bba-2e25-460b-be2e-6da98172c826 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.096993 May 24 10:33:31.606521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 803/1532] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:32.622897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:32.623565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:32.629664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:32.630110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:32.630869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:32.631417 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:32.692419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:32.700389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1d92c4ac-bd28-44f6-96c5-1819e694983d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081476 May 24 10:33:32.700959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 730/1533] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:33.108247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:33:33.108391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:33:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:33:33.717334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:33.717928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:33.724665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:33.725115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:33.725912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:33.726348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:33.789925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:33.798547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fbcf2ff2-1f1b-46a8-9fdf-28c7256a6a10 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085148 May 24 10:33:33.799114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 804/1534] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:34.817548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:34.818378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:34.825093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:34.825565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:34.826326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:34.826755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:34.891154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:34.899143 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e83ee57a-3bff-4874-97e7-651ae970353a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086368 May 24 10:33:34.899701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 731/1535] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:35.916764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:35.917451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:35.924625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:35.925075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:35.925876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:35.926317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:35.928831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=84 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:35.987622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:35.995481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-106f68a5-4026-4d1f-ac7e-2967e3057b1f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082609 May 24 10:33:35.996046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 805/1536] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:37.012873 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:37.013485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:37.016497 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=28 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:37.021439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:37.021892 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:37.022692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:37.023129 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:37.085225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:37.093028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-0afeb65f-8eb2-4877-b542-23299ffaee35 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083950 May 24 10:33:37.093799 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 732/1537] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:38.109002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:38.109659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:38.112016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=28 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:38.116704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:38.117155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:38.117952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:38.118388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:38.179932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:38.187999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd064b1b-b5f0-491d-b415-04d6507cbd65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082948 May 24 10:33:38.188563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 806/1538] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:39.203878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:39.204548 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:39.211057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:39.211505 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:39.212270 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:39.212705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:39.273483 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:39.281333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9b8e3f54-9847-4b35-86ae-1ad5b9ac89da tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081296 May 24 10:33:39.282485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 733/1539] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:40.299115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:40.299841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:40.306069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:40.306523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:40.307281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:40.307714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:40.370248 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:40.379041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-34825bbd-70db-420a-8ff4-a2e21096fa62 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084300 May 24 10:33:40.379613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 807/1540] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:41.396394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:41.396938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:41.403327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:41.403766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:41.404522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:41.404946 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:41.466230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:41.474508 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-64aa2f03-5e42-494d-b23a-06e40ee3f361 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082144 May 24 10:33:41.475074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 734/1541] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:42.491192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:42.491800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:42.498790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:42.499240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:42.500001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:42.500487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:42.564646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:42.572828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bd03098e-c061-4f15-ae81-f0d047c01bd3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085533 May 24 10:33:42.573416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 808/1542] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:43.590204 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:43.590834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:43.598105 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:43.598554 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:43.599313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:43.599742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:43.665656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:43.674252 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5093bb67-6fa7-4adc-ab14-b56f53209396 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088141 May 24 10:33:43.674819 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 735/1543] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:44.691858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:44.692576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:44.699136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:44.699588 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:44.700344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:44.700774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:44.762413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:44.770466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-812cee0d-9a30-4e59-85ab-aa9be9f7458b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083491 May 24 10:33:44.771031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 809/1544] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:45.787053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:45.787638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:45.794156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:45.794603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:45.795365 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:45.795805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:45.858157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:45.866405 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fd2e4201-9240-47ca-bff0-797ac75f84c7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083280 May 24 10:33:45.866972 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 736/1545] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:46.883018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:46.883612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:46.890074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:46.890517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:46.891277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:46.891706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:46.952964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:46.961502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2597fd01-bf97-4682-a884-91264932091d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082303 May 24 10:33:46.962064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 810/1546] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:47.978836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:47.979481 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:47.986778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:47.987209 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:47.987971 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:47.988408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:48.049068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:48.057128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cb149ec5-c1a5-4f3c-be66-016887353a34 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082150 May 24 10:33:48.057741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 737/1547] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:49.073807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:49.074419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:49.081948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:49.082410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:49.083175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:49.083612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:49.144999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:49.153201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-83c9aec7-7947-400f-acf0-f4281daa474c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083311 May 24 10:33:49.154399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 811/1548] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:49.906094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:33:49.906436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:33:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:33:50.170796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:50.183641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:50.183641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:50.183641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:50.183641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:50.184054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:50.241061 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:50.249033 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bf1278f9-dfc5-43b2-9817-ef8369272317 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082105 May 24 10:33:50.249635 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 738/1549] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:51.266223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:51.266863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:51.273723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:51.274166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:51.274924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:51.275347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:51.339732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:51.348242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-912373fe-87f9-4947-97a2-d64ca09347ba tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085872 May 24 10:33:51.348794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 812/1550] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:52.365712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:52.366379 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:52.372903 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:52.373372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:52.374219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:52.374649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:52.439426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:52.447605 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6839a6db-8f95-4b6f-8224-fc0a1e4de0b4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085847 May 24 10:33:52.448169 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 739/1551] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:53.465260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:53.465896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:53.472348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:53.472837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:53.473618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:53.474047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:53.534813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:53.542925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8091ee83-f363-42eb-97fe-3c1a128b2730 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081785 May 24 10:33:53.543500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 813/1552] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:54.559545 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:54.560184 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:54.567128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:54.567575 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:54.568339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:54.568864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:54.629753 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:54.638234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e8f2f854-97ae-41b8-8a95-47f006fd350c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082577 May 24 10:33:54.638793 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 740/1553] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:55.655938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:55.656551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:55.663988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:55.664438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:55.665212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:55.665678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:55.728868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:55.736952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bce1417e-c54d-4b96-8307-1aef5a2b21d1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085001 May 24 10:33:55.737548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 814/1554] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:56.986205 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:56.986842 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:56.993354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:56.993798 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:56.994550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:56.994980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:57.055220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:57.063005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-be72b784-97d4-41e9-990e-8b2e6274e5e3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.313477 May 24 10:33:57.063569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 741/1555] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:58.079377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:58.079993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:58.086519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:58.086969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:58.087737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:58.088173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:58.250088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:58.258893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2563f737-d0c2-4d11-a8f7-97f2655996f9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.183416 May 24 10:33:58.259451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 815/1556] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:33:59.275281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:33:59.275929 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:33:59.282402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:59.282850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:59.283644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:33:59.284075 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:33:59.288333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=90 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:59.347560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:33:59.355326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-11b85d82-313f-4e5c-b9f7-241f8c22b839 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084688 May 24 10:33:59.355885 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 742/1557] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:33:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:00.372570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:00.373171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:00.379839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:00.380789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:00.381291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:00.382408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:00.446305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:00.454293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-608eeaa0-4a61-451e-8654-f5257b74ec0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085851 May 24 10:34:00.454851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 816/1558] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:00.695658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:34:00.696201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:34:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:34:01.471534 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:01.472230 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:01.479062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:01.479504 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:01.480394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:01.480840 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:01.545831 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:01.554056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1482a75a-08b2-4089-ac7e-505ad7e9d214 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086800 May 24 10:34:01.554624 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 743/1559] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:02.575280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:02.575939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:02.582420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:02.582867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:02.583632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:02.584087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:02.644692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:02.653105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9317d6c1-b5fd-4e54-85d3-086e1ee92b56 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081764 May 24 10:34:02.653695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 817/1560] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:03.669691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:03.670406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:03.676677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:03.677120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:03.677901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:03.679277 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:03.739332 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:03.748092 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7e6c0884-039e-49cd-984a-e513186bbbcb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082239 May 24 10:34:03.748657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 744/1561] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:04.763966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:04.764605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:04.771228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:04.771679 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:04.773054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:04.773522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:04.833298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:04.841647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f98eaaa0-20cc-4e95-9ba5-8e13bd51af21 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081553 May 24 10:34:04.842213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 818/1562] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:05.859231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:05.859775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:05.867009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:05.867454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:05.868208 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:05.868638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:05.929744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:05.937606 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4dd92116-2d0e-4496-b654-44a78508c33c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082241 May 24 10:34:05.939735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 745/1563] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:06.955104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:06.955691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:06.962945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:06.963390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:06.964150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:06.964583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:06.968427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=84 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:07.026487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:07.034420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a02162b9-851f-45ad-b96c-ec11a4842762 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083190 May 24 10:34:07.034983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 819/1564] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:08.050961 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:08.051570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:08.054356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=28 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:08.059078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:08.059531 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:08.060296 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:08.060732 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:08.122839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:08.130802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c3025f2f-9d28-4eeb-8064-92ec79ca9f86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083713 May 24 10:34:08.132036 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 746/1565] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:09.148481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:09.149000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:09.150648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=28 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:09.155384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:09.157343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:09.157343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:09.157343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:09.222543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:09.231215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5ccae41a-2e1d-435f-bd51-63c6101cc4cd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087231 May 24 10:34:09.231788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 820/1566] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:10.247955 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:10.248564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:10.255019 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:10.255463 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:10.256220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:10.256718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:10.321415 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:10.329691 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d3fda41f-7c03-4913-90dc-21d3c382f8ce tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085526 May 24 10:34:10.330267 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 747/1567] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:11.347365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:11.347965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:11.354502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:11.354945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:11.355695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:11.356123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:11.416437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:11.424233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fcccb483-4eab-433d-a5ab-c4ddcf5766ee tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080750 May 24 10:34:11.424790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 821/1568] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:12.440301 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:12.440909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:12.447647 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:12.448090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:12.448843 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:12.449286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:12.509764 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:12.517526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e43adeb0-e78e-4a78-9214-2464721c3a42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081073 May 24 10:34:12.518083 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 748/1569] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:13.534563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:13.535269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:13.542218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:13.542672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:13.543443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:13.543882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:13.607138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:13.615024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c5b063c3-7ede-4830-b709-e2214c217afe tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084315 May 24 10:34:13.615601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 822/1570] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:14.631817 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:14.632482 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:14.639065 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:14.639510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:14.640261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:14.640690 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:14.702982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:14.711503 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4ddfd48f-fd40-4ca6-abb1-a4e2f193d589 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083529 May 24 10:34:14.712062 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 749/1571] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:15.727163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:15.727714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:15.734354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:15.734803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:15.736216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:15.736216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:15.796896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:15.805193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-580f2d0c-6aca-4211-a677-c24ad8cd577d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082041 May 24 10:34:15.805794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 823/1572] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:16.822118 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:16.822727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:16.829361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:16.829808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:16.830557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:16.830986 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:16.891278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:16.899907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bcd50076-24b7-4e60-81d0-e08e8843bb5c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081712 May 24 10:34:16.900467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 750/1573] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:17.916538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:17.917180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:17.923899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:17.924352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:17.925119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:17.925585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:17.989134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:17.997916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0297cad2-19fd-43c0-9aa2-66e2b0d6a89b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085870 May 24 10:34:17.998494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 824/1574] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:19.014522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:19.015116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:19.021674 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:19.022754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:19.023615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:19.024048 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:19.087720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:19.095672 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e7cb1e14-9bb2-4bae-8a92-3b7b81f866c3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085027 May 24 10:34:19.096234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 751/1575] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:20.112985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:20.113622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:20.120793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:20.121243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:20.122043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:20.122477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:20.184495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:20.193230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0aebe82b-68c0-479b-9d5d-3b9ac860f625 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084119 May 24 10:34:20.193836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 825/1576] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:21.210615 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:21.211284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:21.217779 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:21.218224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:21.218980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:21.219413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:21.281331 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:21.289474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c65f0ce6-618a-4c44-a828-91b62203a923 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083383 May 24 10:34:21.290035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 752/1577] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:22.308261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:22.308864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:22.315395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:22.315833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:22.316583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:22.317288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:22.379335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:22.387197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4a92a66d-8073-4900-b07c-ef1a76767b78 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082842 May 24 10:34:22.387760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 826/1578] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:23.404215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:23.404727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:23.411149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:23.411589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:23.412339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:23.413024 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:23.474893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:23.482814 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e52fb565-93c4-497a-9a69-1fde6c3d176b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082574 May 24 10:34:23.485633 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 753/1579] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:24.499271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:24.499878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:24.506418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:24.507084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:24.507852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:24.508282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:24.569825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:24.577646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5f564673-c326-4f3c-baf7-7b8b2367dd26 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082178 May 24 10:34:24.578212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 827/1580] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:25.595116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:25.595728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:25.602656 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:25.605490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:25.605490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:25.605490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:25.665001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:25.672883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-50157651-8389-4edc-9629-aa8a7add20e5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081697 May 24 10:34:25.673459 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 754/1581] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:26.690099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:26.690701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:26.698209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:26.698651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:26.699408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:26.699837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:26.761960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:26.769941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-276bbb39-75d9-4f0b-aad2-b981af359cf2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083652 May 24 10:34:26.770535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 828/1582] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:27.786773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:27.787408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:27.794027 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:27.794474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:27.795228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:27.795660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:27.861286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:27.869054 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8d775fa6-dc11-4ea2-afc1-b34888729bf8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086142 May 24 10:34:27.869641 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 755/1583] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:28.885974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:28.886732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:28.893717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:28.894163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:28.894943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:28.895356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:28.958323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:28.966756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-37bbcfdb-f29f-4d4e-b6b9-674a78ea5091 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084959 May 24 10:34:28.967319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 829/1584] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:29.983507 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:29.984730 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:29.991518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:29.992714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:29.993536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:29.994016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:30.057989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:30.063126 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-69bb3780-d945-47fe-b989-befadf69469f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083683 May 24 10:34:30.063702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 756/1585] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:31.080099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:31.080684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:31.087210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:31.087657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:31.088419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:31.088860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:31.149759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:31.157664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-48f7ac63-b9fa-4ea9-83ce-ff15efd5fccd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081477 May 24 10:34:31.158239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 830/1586] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:32.174543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:32.177869 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:32.181484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:32.181937 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:32.182699 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:32.183133 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:32.187004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=90 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:32.244718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:32.252581 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2d5e30a1-9ae9-4cbf-80d2-c105b7cd0272 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082110 May 24 10:34:32.253152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 757/1587] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:33.270083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:33.270689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:33.277970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:33.278413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:33.279165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:33.279592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:33.339441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:33.347170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5aea32e3-035b-426c-9f80-638061c4e972 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080962 May 24 10:34:33.347732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 831/1588] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:34.364744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:34.365385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:34.372792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:34.373239 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:34.374032 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:34.374467 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:34.487744 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:34.496849 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f1e0b101-5461-4b07-9743-001bc98ea1d0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.136177 May 24 10:34:34.497448 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 758/1589] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:35.514129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:35.514732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:35.521307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:35.521754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:35.522559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:35.523009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:35.587303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:35.595259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-defd590b-2050-4681-a0cb-b6d4990398b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085096 May 24 10:34:35.595817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 832/1590] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:36.611861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:36.612471 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:36.619288 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:36.619741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:36.620533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:36.621020 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:36.685156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:36.693413 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4cf8f414-d822-4611-86c2-7f1d038a906b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085349 May 24 10:34:36.693977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 759/1591] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:37.710631 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:37.711262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:37.718310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:37.718757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:37.719518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:37.719949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:37.722470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=84 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:37.781224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:37.789162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-432f3bb3-9507-49fa-9d67-92ab742fd116 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082402 May 24 10:34:37.789763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 833/1592] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:38.807219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:38.811655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:38.811655 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=28 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:38.815003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:38.815445 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:38.816233 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:38.816669 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:38.880381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:38.888700 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5e00e3a9-db6b-4287-8a89-133bdc583d72 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085948 May 24 10:34:38.889338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 760/1593] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:39.905512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:39.906054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:39.908015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=28 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:39.912721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:39.913168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:39.913962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:39.914390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:39.976642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:39.984652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e5db5348-73e6-4c8c-8d38-151c20e681d5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083084 May 24 10:34:39.985217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 834/1594] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:41.002123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:41.002736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:41.009773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:41.010224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:41.010982 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:41.011422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:41.072157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:41.080025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-34c67357-a13d-4065-93af-4f4206ee0538 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081778 May 24 10:34:41.080594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 761/1595] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:42.097223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:42.097800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:42.104488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:42.104930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:42.105716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:42.106150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:42.168099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:42.176730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-08f0fec5-91f7-4d2d-9fdd-11e3a1bc29b8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083494 May 24 10:34:42.177310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 835/1596] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:43.193987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:43.194594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:43.201256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:43.201707 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:43.202524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:43.202958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:43.265806 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:43.273604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3e39cc49-3de9-4ee5-8bf9-fe399b51ad86 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083758 May 24 10:34:43.274167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 762/1597] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:44.291152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:44.291763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:44.298301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:44.298759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:44.299520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:44.299953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:44.373249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:44.381073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0c7c1e6-e534-4972-a659-5c88223960dd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.094125 May 24 10:34:44.381662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 836/1598] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:45.398372 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:45.398910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:45.406599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:45.407046 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:45.407816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:45.408251 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:45.472264 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:45.480924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8fc6d520-b691-4371-a6c0-7c59c2d8100b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086453 May 24 10:34:45.481528 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 763/1599] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:46.498155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:46.498788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:46.505533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:46.505987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:46.506748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:46.507182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:46.568130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:46.576711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2f0af717-3e2b-4b6b-b411-9ebd33016c69 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083098 May 24 10:34:46.577293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 837/1600] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:47.593119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:47.593739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:47.600300 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:47.600741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:47.601523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:47.601953 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:47.667745 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:47.675704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-80e66973-add6-4c79-a853-c6fa76ec5d29 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086495 May 24 10:34:47.676271 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 764/1601] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:48.693179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:48.693815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:48.700553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:48.701309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:48.701816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:48.702259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:48.763929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:48.771853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2dd41cf7-a777-4629-b5f8-4ceca8146873 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082749 May 24 10:34:48.772485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 838/1602] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:49.787841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:49.788432 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:49.795056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:49.795487 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:49.796244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:49.796673 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:49.857552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:49.865423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9c0ebdb3-aef2-4cff-8193-c24b57563059 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081374 May 24 10:34:49.865984 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 765/1603] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:50.882522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:50.883115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:50.890346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:50.890795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:50.891559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:50.891997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:50.952884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:50.961140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e6b7133b-675e-44bd-9f50-9e111ebb6afd tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082493 May 24 10:34:50.961741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 839/1604] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:51.979138 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:51.979754 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:51.986167 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:51.986610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:51.987361 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:51.987793 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:52.048919 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:52.056886 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c8b689b1-c14a-4c88-872b-675ab781a60a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082131 May 24 10:34:52.057473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 766/1605] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:53.073915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:53.074530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:53.081057 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:53.081540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:53.082309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:53.082748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:53.147108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:53.155218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8325fa3f-5a24-4cca-94be-22e7772716bb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085433 May 24 10:34:53.155788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 840/1606] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:54.171800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:54.172416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:54.178888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:54.183692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:54.183692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:54.183692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:54.245327 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:54.253132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-30555ab4-1175-4dae-b48c-c23ec47951a2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085179 May 24 10:34:54.253740 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 767/1607] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:55.271059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:55.271694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:55.278823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:55.279276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:55.280042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:55.280498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:55.340810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:55.348693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a915e5c-f288-4514-8402-4d910fd85b40 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081565 May 24 10:34:55.349260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 841/1608] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:55 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:56.365511 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:56.366378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:56.373427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:56.373882 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:56.374652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:56.375091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:56.443010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:56.455141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-92fe7ee6-8b9d-46ce-9ba6-480b59e43fe3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.093546 May 24 10:34:56.455723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 768/1609] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:34:57.476181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:57.476797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:57.483473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:57.483922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:57.484689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:57.485124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:57.548123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:57.556000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-910c3596-c1b4-4c5e-94c6-efd481a94f18 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083681 May 24 10:34:57.557282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 842/1610] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:58.574112 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:58.574730 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:58.581152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:58.581640 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:58.582406 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:58.582841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:58.646225 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:58.654426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c79cf1af-1f6f-4aad-8e10-3d0952c9e60e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084932 May 24 10:34:58.655192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 769/1611] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:34:59.672093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:34:59.672628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:34:59.681030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:59.681562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:59.682472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:34:59.682911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:34:59.742499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:34:59.750338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-41638ea3-c901-4fd9-9eb5-039a16d3cf60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082229 May 24 10:34:59.750901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 843/1612] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:34:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:00.766958 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:00.767550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:00.774248 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:00.774723 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:00.775733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:00.776275 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:00.838312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:00.846367 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f49bdcbd-9a2b-4595-a0f3-b829335cba1e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083325 May 24 10:35:00.846943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 770/1613] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:01.863880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:01.864522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:01.871670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:01.872122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:01.872886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:01.873346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:01.936838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:01.945004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9f832664-3ace-476b-995b-ac73b603511e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084999 May 24 10:35:01.945605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 844/1614] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:02.435692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:02.436313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:02.443629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:02.444166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:02.444934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:02.445400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:02.449572 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=84 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:02.470959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Going to try to terminate instance {{(pid=118233) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:35:02.784037 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5eedfe4d-ef4b-409a-8c95-33b43a7a66ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 204 len: 0 microversion: 2.1 time: 0.352262 May 24 10:35:02.784627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 771/1615] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:35:02 2022] DELETE /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:35:02.792936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:02.793745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:02.799849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:02.800302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:02.801068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:02.801612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:02.926015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:02.934073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eca4da66-afb0-47c4-bd92-3e5c18bd60a6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145207 May 24 10:35:02.934644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 845/1616] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:02.964090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:02.964920 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:02.971070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:02.971526 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:02.972294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:02.972734 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:03.038228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:03.046539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fb7eda8c-c4ff-44e5-a7bc-e21048d73b87 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087740 May 24 10:35:03.047547 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 772/1617] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:03.952125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:03.952735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:03.959355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:03.959833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:03.960615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:03.961049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:04.063518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:04.064228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:04.070987 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:04.071522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:04.072544 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:04.073006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:04.090820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:04.099045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f7868301-5599-4c16-8571-277f9b9c01a2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150820 May 24 10:35:04.099622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 846/1618] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:03 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:04.133836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:04.142053 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dbae1694-e0be-4a69-8bda-93c1d1b46cc9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082445 May 24 10:35:04.142619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 773/1619] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:05.117508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:05.118191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:05.126043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:05.126487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:05.127245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:05.127673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:05.159247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:05.159922 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:05.167917 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:05.168451 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:05.169221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:05.169677 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:05.232431 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:05.240338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-108553cd-6cfc-4285-a8cb-d369f8138fdf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084997 May 24 10:35:05.240909 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 774/1620] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:05.258787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:05.266887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-54f056a0-136f-4af1-b588-429c83d1ff33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153755 May 24 10:35:05.267443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 847/1621] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:05 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:06.257768 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:06.258378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:06.265549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:06.265992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:06.266784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:06.267217 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:06.282964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:06.283570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:06.290069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:06.290517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:06.291402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:06.291992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:06.331749 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:06.340396 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bbb3761a-76b8-4227-be35-2102942d3254 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086441 May 24 10:35:06.341073 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 775/1622] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:06 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:06.419492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:06.427450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6575c11c-0102-483b-a3c6-a8610858e605 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148344 May 24 10:35:06.428014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 848/1623] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:07.356739 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:07.357374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:07.363968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:07.364414 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:07.365183 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:07.365649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:07.429149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:07.437058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3f98c192-d8df-4527-a91b-78720e3bd433 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084282 May 24 10:35:07.437662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 776/1624] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:07.445317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:07.445964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:07.452880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:07.453367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:07.454137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:07.454573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:07.587014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:07.589815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:07.596095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec09e47c-a8a0-4412-885e-1f7ca1438ee8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154914 May 24 10:35:07.596663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 849/1625] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:07 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:08.453978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:08.454589 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:08.461060 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:08.461536 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:08.462293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:08.463302 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:08.523968 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:08.532381 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2ac0ea9c-3f88-40c6-8f40-9e033fa1bfd4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082278 May 24 10:35:08.532938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 777/1626] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:08.616821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:08.617456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:08.624223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:08.624663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:08.625449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:08.625875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:08.752041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:08.761226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e4a5fde8-1a32-4a59-bd52-110cf6ddc6dd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147996 May 24 10:35:08.761830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 850/1627] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:09.549701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:09.550315 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:09.552777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=41,nova_api:UPDATE=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:09.558299 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:09.558746 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:09.559543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:09.559980 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:09.623411 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:09.631116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23cf0319-b85f-4ef3-a853-76567aabf207 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085373 May 24 10:35:09.631940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 778/1628] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:09.778742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:09.779357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:09.786065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:09.786516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:09.787288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:09.787728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:09.917398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:09.925335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a0eb530-996a-446d-914f-6385f7b02d56 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150752 May 24 10:35:09.925906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 851/1629] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:10.648428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:10.649094 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:10.656141 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:10.656587 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:10.657403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:10.657839 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:10.724334 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:10.733337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-4d286867-04d0-44d2-9dae-53b766d8fbca tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088670 May 24 10:35:10.734228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 779/1630] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:10.942387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:10.943244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:10.945560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=36 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:10.951196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:10.957404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:10.957404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:10.957404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:11.080581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:11.088440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd024e8d-200e-419c-ad85-fbf768b65bb5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149920 May 24 10:35:11.089133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 852/1631] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:10 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:11.750229 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:11.750838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:11.757446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:11.757893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:11.758790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:11.759227 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:11.823040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:11.831775 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-52d58c2d-fa3d-4dea-979c-b192a1a749a6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085403 May 24 10:35:11.832341 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 780/1632] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:12.105939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:12.106494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:12.113236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:12.113714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:12.114482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:12.114920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:12.241940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:12.250711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3466442c-3071-466c-b51b-0742a104d7b4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148754 May 24 10:35:12.251290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 853/1633] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:12 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:12.849200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:12.849803 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:12.856893 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:12.857368 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:12.858120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:12.858549 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:12.918707 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:12.926625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e8d93c29-1da8-426b-b93c-25bb256003ab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081349 May 24 10:35:12.927189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 781/1634] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:13.267894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:13.268501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:13.275043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:13.275492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:13.276883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:13.277610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:13.403610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:13.411532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1df2eb03-a26e-4264-a6c9-e7f6e29f9b6a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147405 May 24 10:35:13.412251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 854/1635] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:13.943813 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:13.944421 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:13.951582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:13.952024 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:13.952784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:13.953216 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:14.015125 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:14.022992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c461fcf0-ef10-4ac5-8b2e-f153b724b863 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083349 May 24 10:35:14.023560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 782/1636] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:14.428732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:14.429298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:14.435751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:14.436198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:14.436955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:14.437421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:14.565703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:14.574010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-faca32a3-729c-46b4-9bc2-ab35ab8da0ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149240 May 24 10:35:14.574572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 855/1637] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:14 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:15.039686 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:15.040290 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:15.047203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:15.047654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:15.048420 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:15.048855 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:15.110857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:15.118811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2d4d79a7-a6a3-4f7a-a768-64e03508364d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082940 May 24 10:35:15.119382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 783/1638] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:15.591091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:15.591697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:15.598171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:15.598612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:15.599365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:15.599791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:15.730965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:15.738995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bfea3307-cc23-4691-b317-a199b4417162 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151738 May 24 10:35:15.739558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 856/1639] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:16.136510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:16.137123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:16.143841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:16.144283 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:16.145039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:16.145504 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:16.211071 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:16.219923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-19ea3c4b-4c18-4887-8f7e-22e511dd6c3a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087664 May 24 10:35:16.222330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 784/1640] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:16.757321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:16.757940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:16.765127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:16.765597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:16.766347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:16.766773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:16.892991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:16.900877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b687a64e-70c6-4917-912d-f66a5a4dfeff tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147456 May 24 10:35:16.901629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 857/1641] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:17.236820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:17.237458 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:17.244152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:17.244668 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:17.246197 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:17.246640 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:17.308374 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:17.316498 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2a8701eb-bdbd-433c-9ab6-5397e46fee17 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083578 May 24 10:35:17.317066 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 785/1642] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:17.918433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:17.919043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:17.925552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:17.926002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:17.926766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:17.927198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:18.054058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:18.062033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b26fa4fa-44a1-4e9a-87de-b937384dab82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147621 May 24 10:35:18.062605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 858/1643] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:17 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:18.333777 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:18.334389 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:18.340856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:18.341338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:18.342107 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:18.342543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:18.345068 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:18.404419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:18.412144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1d6c50a3-d972-4eaa-99e9-f6b39e8ea4d9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082211 May 24 10:35:18.412704 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 786/1644] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:18 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:19.079394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:19.080007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:19.086656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:19.087049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:19.087809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:19.088245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:19.275451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:19.284273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fcd7efa0-73bc-45cd-85a7-1672a008b1d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.208816 May 24 10:35:19.284847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 859/1645] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:19 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:19.428250 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:19.428786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:19.435873 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:19.436348 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:19.437114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:19.437583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:19.498606 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:19.506573 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-74ed0512-8808-497f-bff8-c35e26c16ff5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082618 May 24 10:35:19.507145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 787/1646] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:20.257816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:35:20.258010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Tue May 24 10:35:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:35:20.302492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:20.303120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:20.310458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:20.310906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:20.311668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:20.312101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:20.439311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:20.447430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-70377112-5c51-4e94-a6a9-7bb0c21be61a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148730 May 24 10:35:20.448012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 860/1647] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:20 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:20.523619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:20.524219 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:20.531675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:20.532114 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:20.532872 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:20.533359 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:20.599464 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:20.607995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bdea9c24-660a-49f3-bb0c-e9ceaa5f6b06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088238 May 24 10:35:20.608550 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 788/1648] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:21.463941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:21.464555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:21.471087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:21.471537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:21.472304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:21.472736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:21.603111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:21.611726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c3c8dd3d-0449-4308-ab41-70ef3e8a822c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151670 May 24 10:35:21.612313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 861/1649] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:21 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:21.624283 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:21.624883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:21.631464 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:21.631900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:21.632648 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:21.633077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:21.692960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:21.701005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-ef15bc02-8f19-42ee-8362-66adbce164a4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081112 May 24 10:35:21.701595 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 789/1650] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:22.629551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:22.630164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:22.637365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:22.637862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:22.638632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:22.639067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:22.717975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:22.718567 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:22.725203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:22.725687 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:22.726454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:22.726888 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:22.769394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:22.777524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-20d486fa-d0be-475c-9ce0-97aa13ba7609 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151878 May 24 10:35:22.778110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 862/1651] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:22.789717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:22.797662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-266d1bcc-e108-481f-8a98-1e2e2475a889 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083506 May 24 10:35:22.798212 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 790/1652] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:23.795342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:23.795944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:23.802387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:23.802828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:23.803580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:23.804008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:23.813631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:23.814242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:23.820858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:23.821326 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:23.822090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:23.822517 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:23.884187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:23.892808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-93672f8d-736e-4311-a9e7-0c770d0a9de8 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082870 May 24 10:35:23.893473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 791/1653] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:23.934937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:23.942744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-055493e1-8a5c-4bda-b863-47d782537f11 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151985 May 24 10:35:23.943309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 863/1654] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:23 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:24.909845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:24.910506 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:24.917016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:24.918058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:24.918810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:24.919239 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:24.959791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:24.960401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:24.967549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:24.967996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:24.968765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:24.969198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:24.983462 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:24.991489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2f5e269f-2f78-41e8-9d00-3dc53bc69094 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085529 May 24 10:35:24.992139 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 792/1655] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:25.095763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:25.103741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cc1d68ac-22f6-4f85-8836-5c892ec5c5e6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147767 May 24 10:35:25.104331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 864/1656] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:24 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:26.007899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:26.008534 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:26.015065 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:26.015512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:26.016265 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:26.016693 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:26.077675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:26.085627 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-02ad3c47-a347-49dc-917e-97768cd0894b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081555 May 24 10:35:26.086182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 793/1657] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:26.122223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:26.122832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:26.129236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:26.129704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:26.130456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:26.130885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:26.259746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:26.267691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ecf2fe6a-f0b1-47cf-bfb2-8d29ab78c393 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150076 May 24 10:35:26.268319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 865/1658] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:26 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:26.316656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:35:26.316812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:35:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:35:27.102335 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:27.102943 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:27.109790 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:27.110234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:27.110988 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:27.111416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:27.175587 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:27.183577 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a250f73a-778a-45a3-888c-daabd40ffc6b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085225 May 24 10:35:27.184134 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 794/1659] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:27.284958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:27.285600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:27.292096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:27.292541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:27.293336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:27.293775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:27.308537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:27.428032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:27.433099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3019c91b-5468-4356-8b70-8dc604c7a79d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152216 May 24 10:35:27.433687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 866/1660] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:27 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:28.200254 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:28.201490 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:28.209469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:28.209913 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:28.210665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:28.211097 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:28.271657 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:28.280680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-98838b90-b877-49a4-86c6-9d2643947642 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084208 May 24 10:35:28.281256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 795/1661] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:28.451193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:28.452135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:28.458675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:28.459124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:28.459888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:28.460326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:28.590100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:28.598638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2f26dbab-de1e-49cc-8b40-53274d679324 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151814 May 24 10:35:28.599448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 867/1662] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:28 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:29.298317 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:29.298929 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:29.305594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:29.306047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:29.306820 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:29.307256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:29.373003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:29.381144 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-647082da-c763-4ce3-a74b-06dad9e32006 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087312 May 24 10:35:29.381736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 796/1663] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:29 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:29.615827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:29.616424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:29.623019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:29.623463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:29.624221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:29.624648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:29.751930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:29.759860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef253785-fea7-4917-a8fb-6ff83097961e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147960 May 24 10:35:29.760427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 868/1664] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:29 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:30.398057 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:30.398661 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:30.405558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:30.406005 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:30.406766 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:30.407199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:30.715826 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:30.723883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f2238782-0c58-4ec1-8327-40c82214a23d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.329713 May 24 10:35:30.724468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 797/1665] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:30.777463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:30.778073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:30.785214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:30.785676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:30.786466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:30.786902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:30.915416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:30.923782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ad93996a-fd59-4b6d-a9a8-8f8b2e2d69fa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150225 May 24 10:35:30.924345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 869/1666] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:30 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:31.741231 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:31.741981 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:31.749345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:31.749797 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:31.750557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:31.750995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:31.812093 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:31.820356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a2db2672-5a17-4897-9d8d-26f6f44bb447 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082919 May 24 10:35:31.820922 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 798/1667] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:31.941279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:31.942038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:31.948652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:31.949089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:31.949880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:31.950310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:32.079096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:32.086903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-48d9b775-326c-4128-b3d0-54a930773783 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149464 May 24 10:35:32.087461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 870/1668] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:31 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:32.838136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:32.838755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:32.845997 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:32.846489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:32.847260 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:32.847728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:32.908728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:32.916708 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-114de9dd-8e23-4a77-9c67-4b9680a03451 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082408 May 24 10:35:32.917291 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 799/1669] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:33.103874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:33.104405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:33.111189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:33.111640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:33.112411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:33.112852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:33.238931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:33.246920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cc609fbf-c7e8-4342-a421-1acb6e87ac29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147001 May 24 10:35:33.248014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 871/1670] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:33 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:33.934879 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:33.935871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:33.941784 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:33.942232 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:33.942994 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:33.943430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:34.006845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:34.014856 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-edecdd9c-849c-477a-926c-65bd200c1bda tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085325 May 24 10:35:34.015422 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 800/1671] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:34.264448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:34.265122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:34.271560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:34.272005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:34.272758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:34.273182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:34.401185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:34.409906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8c4c5345-b05d-48ba-aaa4-f085c8de8da0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149869 May 24 10:35:34.410478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 872/1672] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:34 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:35.031544 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:35.032130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:35.038714 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:35.039153 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:35.039901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:35.040328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:35.105312 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:35.113833 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-2a1765a6-2807-47e6-89e8-6a3556cd4e5d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086311 May 24 10:35:35.114393 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 801/1673] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:35.427195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:35.427825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:35.434263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:35.434713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:35.435591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:35.436131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:35.565211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:35.573551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d505260e-3c31-4806-b789-2f24248adccb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150319 May 24 10:35:35.574140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 873/1674] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:35 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:36.131019 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:36.131644 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:36.138791 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:36.139269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:36.140037 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:36.140472 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:36.201258 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:36.203496 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:36.209796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cfdd30a5-9ee4-412a-9e9a-0be82ea38183 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082654 May 24 10:35:36.210366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 802/1675] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:36.590948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:36.591657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:36.598129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:36.598571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:36.599323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:36.599749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:36.726782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:36.735517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-03050aa3-df64-4121-95bb-0c1cd4a29ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148434 May 24 10:35:36.736087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 874/1676] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:36 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:37.226759 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:37.227475 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:37.233974 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:37.235716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:37.235716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:37.235716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:37.296360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:37.304338 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5ca59321-2736-4bc1-894e-0497739e2cb6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081797 May 24 10:35:37.308400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 803/1677] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:37.395338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:35:37.395561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:35:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:35:37.752924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:37.753649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:37.760096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:37.760542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:37.761330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:37.761766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:37.892225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:37.900154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0fee2761-a0b9-41fa-99aa-65b393deda0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151157 May 24 10:35:37.900718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 875/1678] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:37 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:38.321132 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:38.321767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:38.328977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:38.329458 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:38.330213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:38.330643 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:38.390631 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:38.398735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-9e73df4b-72a9-4da5-9ddc-d38aa4c40d65 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081752 May 24 10:35:38.399289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 804/1679] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:38.917700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:38.918326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:38.925140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:38.925623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:38.926419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:38.926856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:39.055005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:39.063172 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-42681f7b-b9f3-40d1-bf1c-f40ae9ecd94d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149625 May 24 10:35:39.063738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 876/1680] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:38 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:39.415808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:39.416349 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:39.423474 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:39.423917 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:39.424666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:39.425097 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:39.491010 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:39.499007 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-491ae55d-62bc-45c5-a060-e795ff0adf3e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087111 May 24 10:35:39.499571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 805/1681] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:40.081018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:40.081637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:40.088052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:40.088493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:40.089244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:40.089694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:40.216695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:40.224732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dc39ae7f-ef3e-466a-ae90-459743c46d4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147582 May 24 10:35:40.225318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 877/1682] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:40 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:40.515582 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:40.516192 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:40.522774 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:40.523223 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:40.523985 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:40.524416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:40.584730 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:40.593202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-98533b98-56cc-4c80-be03-a032cdde95f0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081523 May 24 10:35:40.593800 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 806/1683] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:40 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:41.242418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:41.243023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:41.245409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:41.250203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:41.250644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:41.251401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:41.251831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:41.382460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:41.390833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-484ac88a-56f8-4844-b48b-e1e2136c2e76 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152386 May 24 10:35:41.391394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 878/1684] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:41 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:41.608815 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:41.609446 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:41.611748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=58 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:41.616535 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:41.616978 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:41.617767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:41.618206 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:41.680284 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:41.689495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6889ba11-40bd-4768-b996-bb827c8e8601 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084485 May 24 10:35:41.690072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 807/1685] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:42.408361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:42.408965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:42.415642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:42.416003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:42.416759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:42.417186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:42.544148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:42.552121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d3ae01c2-479a-4ea6-b1af-b46730de1215 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148255 May 24 10:35:42.552684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 879/1686] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:42 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:42.707166 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:42.707771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:42.714211 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:42.714654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:42.715409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:42.715836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:42.778149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:42.786696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-bee4fc08-18cd-4a5e-a4bb-8877cc71392d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083468 May 24 10:35:42.787261 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 808/1687] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:43.569444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:43.570059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:43.576957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:43.582724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:43.582724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:43.582724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:43.803570 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:43.804180 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:43.811598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:43.812040 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:43.812810 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:43.813241 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:43.937783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:43.946539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a955f458-1fae-494b-b2b7-d7084637c1d6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.381188 May 24 10:35:43.947140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 880/1688] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:43 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:44.039747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:44.047838 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7d36bf23-24a3-482c-a1d2-10d5430d58c5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.248332 May 24 10:35:44.048403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 809/1689] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:44.964331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:44.965714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:44.972404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:44.972862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:44.973763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:44.974202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:45.064070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:45.064743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:45.073286 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:45.074109 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:45.074510 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:45.074945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:45.105767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:45.115944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bce0944b-cdb6-475d-86b2-d793ffddac82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155666 May 24 10:35:45.116948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 881/1690] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:44 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:45.141149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:45.151088 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-1790d8dc-04a2-437e-bb2f-c997db5122e9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.091120 May 24 10:35:45.152009 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 810/1691] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:46.133667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:46.134087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:46.141299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:46.141739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:46.142511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:46.142942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:46.169858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:46.171074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:46.178120 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:46.178571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:46.179360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:46.179823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:46.243137 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:46.252382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-f17ad656-ad52-492b-8bc2-003f4bf27eb9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087256 May 24 10:35:46.252947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 811/1692] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:46.347992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:46.356392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-77c1c06b-9bde-4a79-a051-4ddd4c81b84d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.227072 May 24 10:35:46.357504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 882/1693] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:46 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:47.268642 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:47.269256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:47.275712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:47.276154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:47.276914 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:47.277385 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:47.338217 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:47.346130 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-45146237-ebe3-42d5-b98e-4fd0b2f634d6 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081921 May 24 10:35:47.346692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 812/1694] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:47.373968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:47.374605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:47.381009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:47.381503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:47.382258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:47.382688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:47.386506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:47.513337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:47.522261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c726c55e-2c49-4a86-a055-40966a1ae66c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152703 May 24 10:35:47.522837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 883/1695] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:47 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:48.362770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:48.363362 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:48.369898 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:48.370346 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:48.371108 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:48.371543 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:48.434559 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:48.442759 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-66fa23ba-ba2e-4f0f-92ab-269370bad340 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084357 May 24 10:35:48.443322 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 813/1696] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:48.538878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:48.539491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:48.546116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:48.546566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:48.547332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:48.547780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:48.675500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:48.681781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-212fc6a4-3f34-4965-b229-e12532388976 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146750 May 24 10:35:48.682404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 884/1697] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:48 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:49.460366 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:49.460963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:49.468802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:49.469634 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:49.470419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:49.470858 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:49.533933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:49.542087 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7601090d-9a97-4fdd-b88b-7626d8a11fc4 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085610 May 24 10:35:49.542649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 814/1698] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:49.699589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:49.700300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:49.708570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:49.709016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:49.709919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:49.710354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:49.846831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:49.857140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4b8af4c5-93a0-4366-acd5-149f722157eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.161458 May 24 10:35:49.857752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 885/1699] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:49 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:50.559409 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:50.560023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:50.566738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:50.567188 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:50.567947 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:50.568382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:50.630059 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:50.637918 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b257edc7-d7d7-49eb-83b0-ea49ecc0bdea tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082471 May 24 10:35:50.638478 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 815/1700] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:50.875159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:50.875685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:50.882213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:50.882671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:50.883440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:50.883881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:51.011780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:51.019993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-11d29e7e-db61-4d49-adf2-86ea0afc35a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149237 May 24 10:35:51.020566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 886/1701] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:50 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:51.655030 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:51.655830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:51.662110 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:51.662553 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:51.663313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:51.663748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:51.724408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:51.732370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5afaa803-0019-4908-9777-f0955793d217 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081251 May 24 10:35:51.732928 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 816/1702] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:51 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:52.037687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:52.038306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:52.044818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:52.045307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:52.046820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:52.047498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:52.178283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:52.186163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dfc1d703-3944-4b2c-850d-0a3237a114ab tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152295 May 24 10:35:52.186988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 887/1703] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:52 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:52.750324 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:52.750916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:52.757373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:52.757815 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:52.758564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:52.759001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:52.821998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:52.830664 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d0abcc95-faf4-4de1-8d43-11d9fd064b0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084761 May 24 10:35:52.831222 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 817/1704] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:53.203217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:53.203822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:53.210350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:53.210792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:53.211546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:53.211978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:53.340652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:53.349626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-43c6e728-f5f1-4f5b-9c5d-e8a4cf8c172c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150402 May 24 10:35:53.350202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 888/1705] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:53 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:53.848542 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:53.849074 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:53.855750 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:53.856202 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:53.856975 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:53.857437 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:53.917866 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:53.926205 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-33663204-1918-451e-9ba3-d34f00cfc94d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082612 May 24 10:35:53.926767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 818/1706] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:54.367594 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:54.368237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:54.375490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:54.375938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:54.376705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:54.377137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:54.503742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:54.511740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d3d32704-fb68-4553-9be4-5c6727d194b3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148044 May 24 10:35:54.512309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 889/1707] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:54.942187 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:54.942792 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:54.949574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:54.950018 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:54.950773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:54.951200 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:54.965495 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:55.012003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:55.020035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a9f3336e-5c31-412e-9186-2531ef64c8cb tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081645 May 24 10:35:55.020594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 819/1708] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:54 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:35:55.528124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:55.528766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:55.535937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:55.536372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:55.537145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:55.537615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:55.667454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:55.675667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c221f060-72ae-40fe-a2e8-71a0d61965b9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151632 May 24 10:35:55.676237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 890/1709] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:56.036119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:56.036736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:56.043214 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:56.043665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:56.044430 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:56.044864 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:56.105619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:56.114065 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7d08acee-60b4-4010-b019-d807d416c519 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081928 May 24 10:35:56.114623 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 820/1710] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:56 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:56.693357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:56.693963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:56.700628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:56.701075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:56.701868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:56.702298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:56.827244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:56.835689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a7a305b-e3ef-48f5-ae6b-d17498f152ae tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146262 May 24 10:35:56.836253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 891/1711] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:56 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:57.130735 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:57.131269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:57.137721 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:57.138162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:57.138915 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:57.139345 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:57.205662 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:57.213593 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a29e89d8-55e9-4b32-bd67-71451509c72f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086803 May 24 10:35:57.214151 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 821/1712] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:57 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:57.853552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:57.854149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:57.860859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:57.861361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:57.862114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:57.862545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:58.128534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:58.136571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b47639a2-6805-4663-8bc9-447e232b31e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.286891 May 24 10:35:58.137138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 892/1713] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:57 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:58.230996 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:58.231599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:58.238610 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:58.239056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:58.239807 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:58.240235 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:58.299824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:58.308006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e31da681-686b-4dd9-816f-3a38127d9631 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080871 May 24 10:35:58.308580 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 822/1714] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:58 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:59.154091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:59.154640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:59.162296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:59.162754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:59.163524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:59.163963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:59.293066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:59.302364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-956e5def-a219-4827-975e-01936e28558b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152198 May 24 10:35:59.302984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 893/1715] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:35:59.325654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:35:59.326266 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:35:59.333515 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:59.333966 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:59.334748 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:35:59.335164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:35:59.398564 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:35:59.406861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-78ce392f-b72c-496c-a07d-76278a17a9aa tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085055 May 24 10:35:59.407360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 823/1716] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:35:59 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:00.319853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:00.320653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:00.327178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:00.328305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:00.329170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:00.329631 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:00.423182 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:00.423808 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:00.430369 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:00.430824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:00.431590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:00.432025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:00.463774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:00.472790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-04d8a17c-4f57-4907-be4c-0a6c2221db00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.157027 May 24 10:36:00.478370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 894/1717] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:00 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:00.497325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:00.505403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5f32fe45-497e-4e27-b6a7-2ded48294469 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086595 May 24 10:36:00.505990 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 824/1718] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:00 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:01.490267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:01.490881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:01.497456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:01.497904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:01.498665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:01.499097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:01.522371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:01.522989 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:01.529638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:01.530090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:01.530868 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:01.531294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:01.596454 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:01.604540 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d0bc0ea8-da43-4c9d-bd1c-ed7427747d0d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085977 May 24 10:36:01.605123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 825/1719] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:01 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:01.632088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:01.639972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fffb2154-26f9-440f-ab2e-16698a8b8e28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153559 May 24 10:36:01.641247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 895/1720] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:02.620652 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:02.621281 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:02.627694 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:02.628145 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:02.628916 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:02.629382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:02.658675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:02.659409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:02.665983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:02.666429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:02.667285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:02.667723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:02.690805 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:02.699738 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-b1310adc-cbf9-4ab5-9a15-67e5da4ce23b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083099 May 24 10:36:02.700318 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 826/1721] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:02 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:02.796647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:02.804571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7501563b-4656-48d9-84c3-8858bd82772d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150506 May 24 10:36:02.805144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 896/1722] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:03.717116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:03.717771 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:03.725028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:03.725599 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:03.726394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:03.726857 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:03.788303 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:03.796122 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8e2bd8c2-56c4-4835-80d7-49a164445a6c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082962 May 24 10:36:03.796680 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 827/1723] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:03 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:03.821958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:03.822513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:03.829224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:03.829744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:03.830518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:03.830961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:03.961956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:03.970533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eca0fddf-0f4e-4586-bbeb-2aa90a30563a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152708 May 24 10:36:03.971114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 897/1724] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:03 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:04.814171 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:04.814773 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:04.821558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:04.822001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:04.822755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:04.823191 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:04.885902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:04.893811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-23cd9643-269a-4242-9eab-90524f437de3 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084192 May 24 10:36:04.894370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 828/1725] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:04 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:04.987314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:04.987931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:04.994604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:04.995052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:04.995820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:04.996253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:05.132824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:05.140922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f67f4dc8-479a-4440-a373-ca561a13fe5a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.157981 May 24 10:36:05.141861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 898/1726] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:04 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:05.912028 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:05.912629 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:05.919391 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:05.919834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:05.920692 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:05.921119 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:05.986041 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:05.994029 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-019f9d65-15b7-4b19-a6dc-4308788950a7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087308 May 24 10:36:05.994645 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 829/1727] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:05 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:06.158694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:06.159346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:06.165932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:06.166381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:06.167148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:06.167583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:06.296860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:06.299454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:06.306119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-640cd7ed-3cf3-4efc-aeab-3a773f76cbbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151786 May 24 10:36:06.306732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 899/1728] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:07.014069 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:07.014665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:07.021382 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:07.021834 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:07.022603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:07.023045 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:07.084942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:07.092999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e54e7fca-7477-4d12-ae86-822f5f448ea1 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082783 May 24 10:36:07.093587 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 830/1729] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:07 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:07.324058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:07.324601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:07.332442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:07.336259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:07.336259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:07.336259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:07.461598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:07.470390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3c4ef8c6-b136-4d35-a4b7-9ca399fbfe6c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149673 May 24 10:36:07.470970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 900/1730] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:07 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:08.110901 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:08.111504 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:08.118084 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:08.118532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:08.119337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:08.119767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:08.181802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:08.190378 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-33465074-b5e9-4372-ae77-173a6f83ec42 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084116 May 24 10:36:08.190940 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 831/1731] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:08 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:08.487346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:08.487959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:08.494436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:08.494887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:08.495656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:08.496091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:08.624978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:08.632916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd60000d-1808-4a59-b4f7-7da99807de62 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150142 May 24 10:36:08.633529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 901/1732] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:09.207223 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:09.207836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:09.214423 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:09.214876 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:09.215649 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:09.216096 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:09.277364 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:09.286256 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-19cce62f-90d8-439d-8540-0fdb9b343070 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083336 May 24 10:36:09.286825 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 832/1733] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:09 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:09.650496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:09.651114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:09.658350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:09.658788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:09.659570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:09.659996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:09.788105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:09.795952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b3d06fc0-85fe-47ff-b05d-af0458905f1c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149316 May 24 10:36:09.796524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 902/1734] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:10.303266 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:10.303867 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:10.310339 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:10.310781 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:10.311532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:10.311960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:10.377124 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:10.385116 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-00076a4e-e7af-4eba-a7d6-cdd1b7cff4e7 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085732 May 24 10:36:10.385712 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 833/1735] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:10 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:10.813484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:10.814101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:10.820682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:10.824843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:10.824843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:10.824843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:10.949831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:10.958630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a6bc44a7-b369-4a50-b87d-0b26d39c2787 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149764 May 24 10:36:10.959193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 903/1736] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:10 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:11.401934 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:11.402552 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:11.409050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:11.409530 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:11.410293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:11.410727 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:11.471196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:11.479158 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c35d45fb-0a6c-4d66-80c5-311016e7818c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081048 May 24 10:36:11.479718 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 834/1737] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:11 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:11.976486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:11.977086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:11.979449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:11.985467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:11.993056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:11.993056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:11.993056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:12.115889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:12.123776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-84dae06d-2090-4c98-84ce-7857e48408f3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151187 May 24 10:36:12.124340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 904/1738] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:11 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:12.496492 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:12.497099 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:12.499402 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=56 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:12.504177 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:12.504621 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:12.505407 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:12.505841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:12.568363 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:12.576518 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-3b7bfeab-3dab-45a8-ac3f-976211587834 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083865 May 24 10:36:12.577070 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 835/1739] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:12 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:13.141178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:13.141895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:13.148392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:13.148835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:13.149735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:13.150164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:13.280308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:13.288586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd2bc1c4-4063-4d33-9301-fe2295ffdf53 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151509 May 24 10:36:13.289155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 905/1740] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:13.592654 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:13.593280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:13.600356 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:13.600802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:13.601598 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:13.602034 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:13.604594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:13.663942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:13.671984 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-01778121-9a4e-4b20-84af-50eb44222d9e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083272 May 24 10:36:13.673571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 836/1741] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:13 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:14.306722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:14.307363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:14.314609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:14.315055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:14.315812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:14.316243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:14.465316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:14.473479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fca63c20-72b2-4924-98c3-a20786f2c1b6 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.170674 May 24 10:36:14.474054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 906/1742] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:14 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:14.690213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:14.690845 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:14.697685 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:14.698128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:14.698884 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:14.699313 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:14.762512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:14.770762 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-25d3f57b-d60c-4d1a-baa1-d4a345f7a699 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085087 May 24 10:36:14.771328 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 837/1743] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:14 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:15.490461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:15.491076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:15.497565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:15.498016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:15.498773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:15.499211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:15.625555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:15.633420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-becdf91e-4de3-4c09-929d-82a79bfcc218 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146860 May 24 10:36:15.633994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 907/1744] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:15.787670 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:15.788199 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:15.794742 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:15.795193 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:15.795951 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:15.796387 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:15.858675 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:15.866728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8beb62e5-d9c4-41f4-9f82-01a8b49c2c06 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082978 May 24 10:36:15.870078 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 838/1745] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:15 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:16.651423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:16.652060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:16.659205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:16.659653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:16.660418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:16.660929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:16.787766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:16.795721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7ad3964f-bee1-43a0-845d-cd988feed963 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148190 May 24 10:36:16.796295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 908/1746] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:16.885172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:16.885809 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:16.892957 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:16.893534 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:16.894293 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:16.894751 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:16.956289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:16.964135 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e7e35e34-9e24-4a9a-ba20-332dbff824b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082860 May 24 10:36:16.964696 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 839/1747] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:16 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:17.813827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:17.814441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:17.821095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:17.821589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:17.822345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:17.822777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:17.952638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:17.961355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-002ee439-9f98-47e1-85ce-4de8feaf2f15 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151451 May 24 10:36:17.961926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 909/1748] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:17 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:17.981190 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:17.981823 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:17.988427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:17.988871 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:17.989811 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:17.990242 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:18.050468 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:18.058360 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-97c44fe4-20dd-4409-a5be-ec34d42fb366 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080954 May 24 10:36:18.058924 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 840/1749] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:17 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:18.979151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:18.979710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:18.986670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:18.987112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:18.987867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:18.988296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:19.075287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:19.075897 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:19.083172 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:19.083619 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:19.084392 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:19.084824 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:19.116059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:19.124061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d7f73e6b-23e9-4dc1-a1be-0df332a5a450 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148878 May 24 10:36:19.124635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 910/1750] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:18 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:19.151152 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:19.159741 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-47216f8b-1691-4729-a2c4-c3813f1368de tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088308 May 24 10:36:19.160304 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 841/1751] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:19 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:20.141039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:20.141691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:20.148283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:20.148728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:20.149514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:20.149945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:20.176373 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:20.177082 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:20.183887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:20.184408 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:20.192135 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:20.192135 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:20.250050 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:20.258346 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-d3a0d153-6afc-494e-b508-b4bc5f116cab tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.086312 May 24 10:36:20.258933 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 842/1752] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:20 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:20.283407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:20.291295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a623971-3c5d-4064-8ac4-fbde6e2ea71c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154201 May 24 10:36:20.291864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 911/1753] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:20 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:21.275184 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:21.275788 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:21.282563 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:21.283006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:21.283758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:21.284195 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:21.308581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:21.309217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:21.316199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:21.316644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:21.317439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:21.317873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:21.345686 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:21.354013 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-160eb22f-aec1-451d-87a9-ea9d2f9e1a4c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082627 May 24 10:36:21.354584 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 843/1754] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:21 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:21.444426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:21.452493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6eda8296-c7b9-4782-94d3-ecd60d726751 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147781 May 24 10:36:21.453060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 912/1755] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:21 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:22.371438 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:22.372047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:22.378590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:22.379035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:22.379836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:22.380358 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:22.441590 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:22.449717 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e56b6d1a-31c4-44b0-b9ca-87fd40e1addf tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082207 May 24 10:36:22.450280 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 844/1756] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:22 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:22.470020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:22.470709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:22.477324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:22.477777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:22.478538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:22.478972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:22.607282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:22.616040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a1cc8c5a-ee0a-43a6-9e05-f4a8ac7162a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150615 May 24 10:36:22.616617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 913/1757] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:23.467004 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:23.467604 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:23.474168 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:23.477294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:23.477294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:23.477294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:23.540702 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:23.548665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-c04e9819-dbe1-495d-bc68-2b02ceb8bb8d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085500 May 24 10:36:23.549247 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 845/1758] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:23 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:23.632872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:23.633522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:23.640175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:23.640631 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:23.641698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:23.642148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:23.775025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:23.783664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c396ccd9-08c4-45ec-bcb2-c33e026e8c26 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154654 May 24 10:36:23.784240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 914/1759] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:23 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:24.566977 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:24.567574 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:24.574337 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:24.574786 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:24.575555 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:24.575992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:24.636484 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:24.645148 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-5a141743-f2a6-4ec2-85ee-6ac32ff377f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083699 May 24 10:36:24.645743 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 846/1760] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:24 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:24.800123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:24.800727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:24.807144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:24.807600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:24.808362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:24.808810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:24.933981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:24.942542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d106f6a1-3f38-47be-8a66-77b9d65f2530 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146561 May 24 10:36:24.943120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 915/1761] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:24 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:25.662660 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:25.663273 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:25.669830 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:25.670278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:25.671039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:25.671473 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:25.732533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:25.740603 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-6363d612-395a-44b3-a4ad-b2c5d7f4f38c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081949 May 24 10:36:25.741163 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 847/1762] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:25 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:25.959873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:25.960485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:25.967055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:25.967496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:25.968251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:25.968683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:25.983703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:26.100306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:26.108294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-419dada5-b0b3-483a-bbcd-1e606ebfc9ca tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152436 May 24 10:36:26.108936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 916/1763] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:25 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:26.757025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:26.757836 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:26.764249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:26.764697 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:26.765560 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:26.766521 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:26.829215 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:26.837080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-62217f62-20af-491b-afde-78bb03ff206c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083881 May 24 10:36:26.837679 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 848/1764] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:26 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:27.127168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:27.127697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:27.134115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:27.134560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:27.135316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:27.135745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:27.265033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:27.273188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1375809d-c91a-4639-b5af-4b2c69166b0f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150728 May 24 10:36:27.273783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 917/1765] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:27 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:27.854289 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:27.854899 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:27.861571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:27.862023 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:27.862789 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:27.863249 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:27.926539 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:27.934489 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-00eb00b2-ccff-4f69-9408-55805c215c74 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084134 May 24 10:36:27.935049 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 849/1766] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:27 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:28.290866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:28.291473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:28.298610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:28.299056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:28.299828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:28.300298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:28.429257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:28.437289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0839a66a-4fc7-45a2-9740-40f88cdad137 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150290 May 24 10:36:28.438604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 918/1767] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:28 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:28.951938 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:28.952587 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:28.959705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:28.960149 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:28.960926 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:28.961383 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:29.022082 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:29.029861 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-728cf153-cd78-4a5a-9a6a-7fff6eb82e11 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081808 May 24 10:36:29.030416 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 850/1768] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:28 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:29.455953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:29.456559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:29.463086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:29.463537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:29.464303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:29.464737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:29.591170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:29.599277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8213d454-1d24-40d8-936f-c9a2e3b1ce88 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147646 May 24 10:36:29.599660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 919/1769] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:29 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:30.046900 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:30.048330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:30.055016 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:30.055469 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:30.056234 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:30.056725 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:30.118101 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:30.126666 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-e8ed6b9e-e2c5-4efe-8dd8-3215180b69f5 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083561 May 24 10:36:30.127244 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 851/1770] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:30 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:30.616529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:30.617135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:30.623692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:30.624136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:30.624891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:30.625968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:30.751838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:30.759810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ad4d971d-8ea4-4e93-989b-b76485df04d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147150 May 24 10:36:30.760379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 920/1771] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:30 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:31.144455 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:31.242614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:31.242614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:31.242614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:31.242614 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:31.244923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:31.244923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:31.244923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:31.244923 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-a4905310-42f9-4d50-9e22-b733feb83b03 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.092293 May 24 10:36:31.245340 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 852/1772] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:31 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:31.776905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:31.777539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:31.784610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:31.785055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:31.785851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:31.786287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:31.925996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:31.933885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-797bdac8-dad3-490c-92e4-1e59fcbacafb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.160856 May 24 10:36:31.934452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 921/1773] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:31 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:32.247945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:32.248558 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:32.255881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:32.256329 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:32.257091 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:32.257565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:32.320532 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:32.328428 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fb01218e-2d03-4e89-8be0-07f71f686373 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084423 May 24 10:36:32.328998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 853/1774] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:32 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:32.951085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:32.951704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:32.958432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:32.958884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:32.959646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:32.960078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:33.087509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:33.096319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-65ec0796-1c09-44db-a013-bd9ab93c502c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148910 May 24 10:36:33.096895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 922/1775] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:32 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:33.345015 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:33.346274 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:33.352755 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:33.353203 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:33.354080 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:33.354516 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:33.415770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:33.424019 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-547f809c-b2a4-498c-895f-7fbe42ffcc89 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082652 May 24 10:36:33.424636 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 854/1776] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:33 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:34.113185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:34.113819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:34.120383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:34.120829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:34.121621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:34.122053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:34.263975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:34.272835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-deb447ba-17fe-4a91-ac78-227ab16cf430 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.163637 May 24 10:36:34.273450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 923/1777] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:34 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:34.440778 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:34.441426 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:34.448544 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:34.449000 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:34.449802 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:34.450243 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:34.514665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:34.523634 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-043c4509-210b-44ba-a0c8-26fa3fa63402 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087611 May 24 10:36:34.524196 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 855/1778] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:34 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:35.291616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:35.292324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:35.299747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:35.300203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:35.300968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:35.301437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:35.425885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:35.433735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c9c48d31-5601-4ae1-ba82-30863fd7f493 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146097 May 24 10:36:35.434311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 924/1779] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:35 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:35.540128 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:35.540733 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:35.547194 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:35.547638 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:35.549006 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:35.549485 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:35.610388 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:35.618287 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7524b632-6a09-4a98-878f-434d3d02b55b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082028 May 24 10:36:35.618846 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 856/1780] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:35 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:36.450928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:36.451538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:36.458676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:36.459124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:36.459881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:36.460314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:36.584943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:36.592834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec26a467-a35d-4ee9-b979-3a8875096b28 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145747 May 24 10:36:36.593426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 925/1781] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:36 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:36.634724 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:36.635330 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:36.641854 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:36.642883 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:36.643850 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:36.644269 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:36.708131 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:36.715965 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-cb3858db-7ea4-4562-aff2-02cd48c8d349 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085074 May 24 10:36:36.716527 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 857/1782] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:36 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:37.610321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:37.610970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:37.618383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:37.618838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:37.619637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:37.620076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:37.733370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:37.733983 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:37.741039 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:37.741512 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:37.742278 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:37.742716 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:37.751222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:37.759342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0b48d9bb-4b4f-4b21-893e-b174e5a838f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152940 May 24 10:36:37.759923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 926/1783] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:37 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:37.809077 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:37.817058 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7b8a7130-02cb-4d64-9b9c-8b1106906a60 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087595 May 24 10:36:37.817658 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 858/1784] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:37 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:38.776661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:38.777294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:38.783432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:38.783876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:38.784682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:38.785115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:38.834001 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:38.834612 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:38.841720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:38.841720 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:38.842325 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:38.842770 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:38.907047 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:38.915942 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dc8d2f38-72e2-4349-9ba8-d473f47108c2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.085917 May 24 10:36:38.916523 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 859/1785] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:38 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:38.919612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:38.928156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-15c4d408-4d92-43fa-9e42-8621a2fa3d33 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155405 May 24 10:36:38.930839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 927/1786] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:38 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:39.933701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:39.934318 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:39.941583 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:39.942189 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:39.942963 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:39.943400 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:39.946670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:39.947278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:39.953656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:39.954099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:39.954851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:39.955281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:40.005908 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:40.013902 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7d06cdb6-d34f-4abb-adb5-52196a9795fc tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.084072 May 24 10:36:40.014701 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 860/1787] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:39 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:40.088244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:40.096238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d0df229f-c300-47a5-9a1b-be55dce450fc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153582 May 24 10:36:40.096888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 928/1788] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:39 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:41.030336 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:41.030945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:41.037394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:41.037841 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:41.038594 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:41.039025 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:41.105991 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:41.114579 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-39096d6e-40e1-4d34-8367-c5e745502775 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.087428 May 24 10:36:41.115156 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 861/1789] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:41 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:41.124999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:41.126233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:41.134725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:41.135164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:41.135920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:41.136351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:41.262701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:41.271140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0b69036e-cc00-4331-b410-6e2134ad4fe8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153440 May 24 10:36:41.271750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 929/1790] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:41 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:42.138752 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:42.139318 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:42.146769 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:42.147221 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:42.147995 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:42.148427 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:42.209945 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:42.217853 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-8164a7c8-c56b-44c1-8ae9-91a70a65025f tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083059 May 24 10:36:42.218419 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 862/1791] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:42 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:42.287968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:42.288651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:42.291079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:42.295814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:42.296262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:42.297020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:42.297483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:42.428014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:42.435975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd39204b-bd06-47a0-abfc-a5b0c0aea58a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151855 May 24 10:36:42.436537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 930/1792] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:42 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:43.234954 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:43.235565 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:43.238012 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_api:SELECT=56 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:43.242992 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:43.243453 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:43.244201 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:43.244632 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:43.306220 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:43.314052 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-be7085cc-a54d-4475-ad71-8b56530a1d38 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083005 May 24 10:36:43.314613 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 863/1793] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:43 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:43.453895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:43.454508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:43.461721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:43.462173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:43.462937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:43.463366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:43.589915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:43.597911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8f076580-5eef-4f23-95a0-1551d4521ae5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147865 May 24 10:36:43.598472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 931/1794] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:43 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:44.331354 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:44.331998 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:44.338921 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:44.339397 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:44.340157 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:44.340588 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:44.401524 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:44.409486 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-11422c3b-618d-4dfd-8cf7-7f731231c55c tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081939 May 24 10:36:44.410056 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 864/1795] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:44 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:44.615386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:44.615999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:44.622546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:44.622993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:44.623749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:44.624178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:44.751255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:44.759383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c3aed4c5-6da0-4c96-ac2a-1de5cfd35a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148150 May 24 10:36:44.759943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 932/1796] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:44 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:45.427910 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:45.428878 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:45.436698 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:45.437154 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:45.437960 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:45.438403 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:45.507865 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:45.515736 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-58a19d07-bf3c-459a-b99a-5d1743f1ad1d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.093232 May 24 10:36:45.516294 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 865/1797] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:45 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:45.776719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:45.777354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:45.783820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:45.784350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:45.785028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:45.785497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:45.788497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:45.918260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:45.926314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9ecdd506-c6a8-4d3b-89ef-9f650ef908d5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153610 May 24 10:36:45.926889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 933/1798] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:45 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:46.532907 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:46.533533 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:46.540072 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:46.540522 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:46.541304 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:46.541747 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:46.603181 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:46.611164 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-82764672-5c1a-4ee8-9ad7-cec0e363364b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.082251 May 24 10:36:46.611728 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 866/1799] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:46 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:46.944693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:46.945340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:46.952013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:46.952465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:46.953230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:46.953689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:47.155227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:47.164418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-51a789ff-2c39-4693-ad64-a9c954e4383b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.224237 May 24 10:36:47.165220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 934/1800] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:46 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:47.627829 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:47.628439 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:47.635038 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:47.635482 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:47.636236 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:47.636665 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:47.697123 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:47.704925 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-fd19a9dd-5e74-4c2f-ad9d-335e33b0f134 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.080967 May 24 10:36:47.705519 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 867/1801] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:47 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:48.181530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:48.186574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:48.188553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:48.188998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:48.189785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:48.190221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:48.323930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:48.331775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-25bc27bd-88bd-40ba-a1d4-68a9ca516541 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154181 May 24 10:36:48.332332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 935/1802] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:48 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:48.722302 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:48.722950 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:48.729228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:48.729705 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:48.730470 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:48.730906 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:48.793571 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:48.801459 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7b602db4-e9c7-4016-9f2c-ddfd98f5f10a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.083035 May 24 10:36:48.802035 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 868/1803] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:48 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:49.349890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:49.350503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:49.356911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:49.357513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:49.358270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:49.359367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:49.488612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:49.497042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fbd48113-da0e-4d2e-90ec-77f454ab6fa9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151511 May 24 10:36:49.497806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 936/1804] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:49 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:49.818881 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:49.819433 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:49.826370 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:49.826816 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:49.827569 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:49.827999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:49.843136 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG dbcounter [-] [118233] Writing DB stats nova_cell1:SELECT=100 {{(pid=118233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:49.894502 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:49.902394 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-dab94368-bd2e-4148-b52b-c83ebff6f3c0 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.088145 May 24 10:36:49.902959 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 869/1805] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:49 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:50.514815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:50.515419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:50.522595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:50.523037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:50.523793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:50.524222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:50.651193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:50.659039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-723d871e-d53c-46fc-b8cd-75ca0fb47a7e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148066 May 24 10:36:50.659608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 937/1806] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:50 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:50.918625 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:50.919162 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:50.925601 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:50.926042 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:50.926796 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:50.927371 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:50.988090 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:50.995999 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-7b595ffb-3c92-4192-8172-7a3f4f457d6e tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.081328 May 24 10:36:50.996557 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 870/1807] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:50 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:51.676748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:51.677352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:51.684617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:51.685063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:51.685864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:51.686294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:51.829434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:51.839198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c82ee9e8-75d9-44f4-9ab3-06f61c57dbc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.166350 May 24 10:36:51.839781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 938/1808] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:51 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:52.012611 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:52.013224 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.compute.api [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118233) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:52.021544 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:52.022731 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:52.023887 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:52.024333 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG oslo_concurrency.lockutils [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118233) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:52.090758 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.policy [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118233) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:52.099003 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: INFO nova.api.openstack.requestlog [None req-56a45948-92a4-4fe6-b6f3-5f2b99a1efe2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.090215 May 24 10:36:52.100213 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: [pid: 118233|app: 0|req: 871/1809] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:52 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:52.856179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:52.856783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:52.863521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:52.864064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:52.865472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:52.865957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:52.992745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:53.000685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-65f43a51-16d7-42ef-8017-c02c51a95a14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148450 May 24 10:36:53.001251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 939/1810] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:52 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:53.118228 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: DEBUG nova.api.openstack.wsgi [None req-16a10262-1782-489c-aaf9-d2af90e12b80 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118233) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:53.133622 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: Traceback (most recent call last): May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: timer() May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: cb(*args, **kw) May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: File "/usr/local/lib/python3.8/dist-packages/eventlet/semaphore.py", line 147, in _do_acquire May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: waiter.switch() May 24 10:36:53.133767 dv-cinder-843118-2-iscsi devstack@n-api.service[118233]: greenlet.error: cannot switch to a different thread May 24 10:36:54.019053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:54.019669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:54.027288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:54.027741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:54.028541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:54.028979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:54.162330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:54.170242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dca6bb90-5e8a-4266-a7f1-8e6005da8e4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155146 May 24 10:36:54.170861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 940/1811] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:36:55.187593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:55.192322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:55.195838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:55.196551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:55.197337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:55.197777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:55.323400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:55.331347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-29b5caad-04b7-43be-ad18-26ee1864c94c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147624 May 24 10:36:55.331913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 941/1812] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:56.347979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:56.348675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:56.355966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:56.356446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:56.357215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:56.357687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:56.492029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:56.500351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-caea0807-4ece-4e5f-a5be-bfd696a32b19 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.156352 May 24 10:36:56.500920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 942/1813] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:56 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:57.517722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:57.518379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:57.525037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:57.525531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:57.526294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:57.526731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:57.651975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:57.659906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b73605f0-9fa3-4c9b-8ef4-5f28600cfb97 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146054 May 24 10:36:57.660478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 943/1814] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:57 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:58.676838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:58.677575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:58.684818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:58.685344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:58.686104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:58.686531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:58.819682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:58.827871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cdbb55d5-10d9-44a7-a4c3-7b33de8c912d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154962 May 24 10:36:58.828568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 944/1815] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:58 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:36:59.845249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:36:59.845911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:36:59.853192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:59.853712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:59.854490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:36:59.854949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:36:59.990799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:36:59.995079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c45d970b-914a-4a62-a075-de769a172528 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153689 May 24 10:36:59.995649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 945/1816] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:36:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:01.011522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:01.012153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:01.018528 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:01.018971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:01.019810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:01.020159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:01.144435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:01.152821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cf4b03ac-5e4c-479b-ba2e-f933aeef28e4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145185 May 24 10:37:01.153406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 946/1817] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:02.169924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:02.170555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:02.177106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:02.177582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:02.178945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:02.179375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:02.309595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:02.317369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d372c9c1-ffb9-4fb5-9d99-b8c85d5fb291 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151268 May 24 10:37:02.317943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 947/1818] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:03.345705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:03.346310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:03.352856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:03.353327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:03.354096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:03.354529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:03.512206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:03.520240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4bf94930-ada5-460e-a6f9-5a4e165edc2d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.178340 May 24 10:37:03.520818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 948/1819] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:03 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:04.538875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:04.539519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:04.546081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:04.546536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:04.547301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:04.547739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:04.680861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:04.683096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:04.689862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7db6ed15-c44c-4276-9977-057d59d3938b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155792 May 24 10:37:04.690449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 949/1820] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:04 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:05.708705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:05.709370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:05.715954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:05.716396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:05.717156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:05.717609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:05.842637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:05.850492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a6fb1627-1644-40aa-9d26-4c46024725fe tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146001 May 24 10:37:05.851057 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 950/1821] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:05 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:06.868262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:06.868983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:06.875400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:06.875851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:06.876613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:06.877050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:07.006957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:07.015835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-13d07dff-f53d-42b2-85a8-13a1dd6fc414 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151773 May 24 10:37:07.016412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 951/1822] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:08.033052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:08.033880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:08.040854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:08.041316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:08.042078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:08.042503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:08.170103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:08.179701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e9fc0dd0-f35c-40ce-b557-f010b792cf44 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148969 May 24 10:37:08.179701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 952/1823] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:09.196861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:09.197530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:09.203656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:09.204100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:09.204854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:09.205307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:09.337618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:09.345609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a49331b5-1359-41f6-a6e6-8cfdffab97ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154056 May 24 10:37:09.346182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 953/1824] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:09.916129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:37:09.916337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:37:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:37:10.362979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:10.363640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:10.370874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:10.371324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:10.372193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:10.372815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:10.502293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:10.510856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2be44186-46db-4eb7-a1ac-99020c194c23 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151738 May 24 10:37:10.511437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 954/1825] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:10 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:11.528122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:11.528808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:11.535497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:11.535951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:11.536720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:11.537153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:11.663692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:11.671572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-db56b419-9264-41cd-96c0-be9424a0372a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147410 May 24 10:37:11.672145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 955/1826] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:11 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:12.689868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:12.690618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:12.692870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:12.698424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:12.698873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:12.699638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:12.700073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:12.827251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:12.835156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3b9ae6f3-3148-4bf2-9d76-f91b738150a5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149174 May 24 10:37:12.836352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 956/1827] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:12 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:13.851953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:13.852563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:13.859047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:13.859490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:13.860242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:13.860673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:13.989997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:13.998444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c79b8dae-dc18-4723-8541-032daa7599bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150336 May 24 10:37:13.999019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 957/1828] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:15.016192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:15.016840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:15.023379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:15.023830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:15.024592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:15.025024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:15.155258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:15.163360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c25ebb4b-7525-4a9b-a296-ea172a63a4f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151035 May 24 10:37:15.163946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 958/1829] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:16.180807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:16.181442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:16.187967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:16.188411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:16.190406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:16.190406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:16.323741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:16.331701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e03cb5dd-2c20-4ec9-b415-00a1e03e3dac tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155227 May 24 10:37:16.332305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 959/1830] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:17.349014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:17.349662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:17.356156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:17.356601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:17.357387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:17.357821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:17.487078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:17.495480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2ed2916d-103f-4224-a7a8-f018ee8308d7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150317 May 24 10:37:17.496601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 960/1831] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:17 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:18.514222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:18.514883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:18.521359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:18.521808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:18.522568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:18.523018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:18.871617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:18.879526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f6cf9525-ec9f-4532-86fc-7d94eaceea55 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.370671 May 24 10:37:18.880094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 961/1832] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:18 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:19.896844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:19.897556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:19.904393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:19.904836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:19.905617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:19.906048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:20.031641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:20.040296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2d3de91a-715b-4dbd-9321-df071845d2f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147245 May 24 10:37:20.040863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 962/1833] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:19 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:21.058235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:21.058933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:21.065540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:21.065990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:21.066751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:21.067185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:21.198073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:21.205986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2e64ad33-7adb-4df4-8760-840b133ebaf7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152384 May 24 10:37:21.206552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 963/1834] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:21 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:22.223669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:22.224307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:22.230909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:22.231411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:22.232175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:22.232607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:22.357011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:22.364852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e73df423-175b-4047-aad4-b75d2dbc33cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145141 May 24 10:37:22.365485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 964/1835] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:23.382280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:23.382974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:23.389832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:23.390280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:23.391041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:23.391473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:23.521997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:23.530811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c9168e8e-5793-4ad5-b94e-c5d490fe1290 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152323 May 24 10:37:23.531380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 965/1836] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:23 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:24.548062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:24.548734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:24.555367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:24.556628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:24.557513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:24.557957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:24.575590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:24.686540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:24.694493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b8e71bce-ce27-473f-b3e2-741a22b26102 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150347 May 24 10:37:24.695070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 966/1837] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:24 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:25.711818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:25.712488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:25.718851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:25.719295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:25.720046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:25.720479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:25.847662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:25.855911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eabb5542-8b5a-46c5-9a97-4ac8569f87a8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147947 May 24 10:37:25.856483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 967/1838] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:25 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:26.872967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:26.873696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:26.880196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:26.880674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:26.881467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:26.881901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:27.009854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:27.017744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8e8eabd3-500e-4514-b2b8-b232ff1f5568 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148722 May 24 10:37:27.018452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 968/1839] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:26 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:28.035917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:28.036616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:28.043719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:28.044193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:28.044946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:28.045478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:28.172397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:28.180472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-610c4d03-57b2-40ea-97e6-de51efa15b35 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148460 May 24 10:37:28.181041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 969/1840] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:28 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:29.198059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:29.198894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:29.205220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:29.205705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:29.206475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:29.206910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:29.340199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:29.348197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-27e630e9-cfec-4003-b46e-5912d40ee273 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154604 May 24 10:37:29.348771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 970/1841] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:29 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:30.365923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:30.366545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:30.373633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:30.374081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:30.374836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:30.375267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:30.502942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:30.511753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8a37b8c8-5e11-417b-afbb-fc9db834115d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149689 May 24 10:37:30.512322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 971/1842] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:30 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:31.529943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:31.530630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:31.537244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:31.537732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:31.538494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:31.538924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:31.669668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:31.677678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cb250433-7ef0-44eb-a82c-c5b0fc0425d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152143 May 24 10:37:31.678255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 972/1843] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:31 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:37:32.695238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:32.695933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:32.703217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:32.703656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:32.704406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:32.704835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:32.829361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:32.837195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dcf6a719-ef62-4bc2-9f8b-4b8f85ce6c34 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145785 May 24 10:37:32.837785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 973/1844] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:32 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:33.855202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:33.855875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:33.862348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:33.862796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:33.863549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:33.863975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:33.991870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:34.000324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-68980e72-6ed0-41c7-b0ac-307ae21d92e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148954 May 24 10:37:34.000890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 974/1845] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:33 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:35.018000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:35.018692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:35.025131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:35.025616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:35.026384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:35.027466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:35.158663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:35.166715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c0471f4e-8b93-43a4-94ec-989eb3a90525 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152549 May 24 10:37:35.167317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 975/1846] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:35 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:36.184138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:36.184759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:36.191302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:36.191769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:36.192520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:36.192947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:36.318095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:36.326308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b64ce6c6-decc-423b-ba6f-25eb4fa08ba5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146305 May 24 10:37:36.326874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 976/1847] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:36 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:37.342713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:37.343367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:37.349896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:37.350343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:37.351108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:37.351543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:37.483744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:37.491701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9f6f211b-a849-4a99-b5e3-05d2509b1b05 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152866 May 24 10:37:37.492273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 977/1848] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:37 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:38.509530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:38.510270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:38.517239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:38.517718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:38.518479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:38.518911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:38.644388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:38.652260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8de3e262-7cdd-4c70-967d-2466de34d238 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146596 May 24 10:37:38.652829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 978/1849] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:38 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:39.668670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:39.669375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:39.675806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:39.681455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:39.681455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:39.681455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:39.809981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:39.817894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ccc793db-30ad-4811-881e-3c524f34047e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153398 May 24 10:37:39.818469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 979/1850] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:39 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:37:40.835966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:40.836669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:40.843247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:40.843692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:40.844452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:40.844882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:40.971197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:40.979167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-59ce7697-9185-418a-bacb-16331b0239f0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147115 May 24 10:37:40.980315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 980/1851] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:40 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:41.995994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:41.996614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:42.003136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:42.003584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:42.004343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:42.004777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:42.130397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:42.139244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0b9b68c7-92e2-46af-af93-9431ab0ef628 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147217 May 24 10:37:42.139823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 981/1852] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:41 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:43.157076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:43.157737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:43.159953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:43.170974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:43.170974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:43.170974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:43.170974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:43.298187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:43.306159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b2646b82-9f05-4788-9b6c-2a12e8aeb8a3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153068 May 24 10:37:43.306762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 982/1853] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:43 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:44.323595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:44.324268 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:44.330798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:44.331250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:44.332002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:44.332431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:44.335103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:44.459688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:44.468279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cefeb104-803b-44a0-a80b-0d0d10f24231 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149019 May 24 10:37:44.468844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 983/1854] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:44 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:45.488052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:45.488825 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:45.495297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:45.496276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:45.497037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:45.497504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:45.632970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:45.643590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-83881670-8f44-46a9-ba7c-e628ee9a056f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.159119 May 24 10:37:45.645725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 984/1855] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:45 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:37:46.661820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:46.662500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:46.669345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:46.669792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:46.670553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:46.670981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:46.804224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:46.812656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d9c74d41-6636-4e0b-ae96-01f675bb5b89 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155263 May 24 10:37:46.813237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 985/1856] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:46 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:47.829742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:47.830436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:47.836933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:47.837449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:47.838216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:47.838645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:47.968153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:47.976318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8b553881-ece2-4b30-bcd3-6457dbb122e1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150401 May 24 10:37:47.976894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 986/1857] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:47 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:48.994272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:48.995034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:49.002055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:49.002512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:49.003279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:49.003718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:49.130333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:49.139343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec3daf42-a71d-4d07-b40a-664024004652 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148603 May 24 10:37:49.143829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 987/1858] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:48 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:50.156972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:50.157793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:50.164233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:50.164683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:50.165481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:50.165922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:50.295023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:50.303943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1af94e1-ed24-4bf8-912a-32d085275ad5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151403 May 24 10:37:50.304530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 988/1859] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:50 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:51.321928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:51.322838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:51.329550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:51.330001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:51.330761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:51.331198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:51.462504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:51.471125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e47e96d0-45ad-4f3a-b95a-96222d462b2c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153113 May 24 10:37:51.471706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 989/1860] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:51 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:52.488564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:52.489249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:52.495659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:52.496110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:52.496863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:52.497312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:52.626834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:52.634823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-00d788df-ea87-4458-bd6f-404d1ab0a02e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150231 May 24 10:37:52.635400 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 990/1861] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:52 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:53.178397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:53.179034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 79059316-a25c-4dd7-954c-0f7a413203a1] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:53.185128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:53.186545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:53.187423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:53.187863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:53.259931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:53.269050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1ee18826-e286-484f-aba6-de63def4458d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1" status: 200 len: 1375 microversion: 2.1 time: 0.094429 May 24 10:37:53.270039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 991/1862] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:53 2022] GET /compute/v2.1/servers/79059316-a25c-4dd7-954c-0f7a413203a1 => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:53.285877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d703e569-94fc-48b6-9f6c-c471894d45b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '.version_select at 0x7f873752d430>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:53.361220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d703e569-94fc-48b6-9f6c-c471894d45b9 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1192980076" status: 202 len: 0 microversion: 2.1 time: 0.078906 May 24 10:37:53.362188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 992/1863] 10.222.0.35 () {58 vars in 1220 bytes} [Tue May 24 10:37:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1192980076 => generated 0 bytes in 80 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:37:53.652675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:53.653409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:53.660962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:53.661527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:53.662397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:53.662891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:53.812212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:53.820973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d0b110a2-ed52-49a9-98a9-a2e22dab6594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.172816 May 24 10:37:53.821609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 993/1864] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:53 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:54.839634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:54.840334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:54.847331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:54.847779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:54.848538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:54.848972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:54.974350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:54.982801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-be1bd2f3-38c2-4f92-9ed3-959fe69b88f4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146459 May 24 10:37:54.983382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 994/1865] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:56.001029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:56.001766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:56.008139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:56.008614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:56.009397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:56.009829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:56.141570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:56.149639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6b13e1d6-fcb3-4c52-ba36-2ae2ed597e5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153368 May 24 10:37:56.150242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 995/1866] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:57.166871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:57.167538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:57.174534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:57.174980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:57.175739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:57.176174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:57.303091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:57.311757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-99a1668f-dff7-4736-bf1b-a540af8f2831 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148683 May 24 10:37:57.312336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 996/1867] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:57 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:58.329752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:58.330376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:58.337019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:58.337500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:58.338254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:58.338685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:58.469796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:58.477899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a2a66cff-2b19-4dc0-88ee-37f4f024dfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152688 May 24 10:37:58.478469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 997/1868] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:58 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:37:59.495231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:37:59.495868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:37:59.502554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:59.502993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:59.503745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:37:59.504171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:37:59.630617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:37:59.639007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-65d103f3-98ad-41ef-9d1c-207dcf610a50 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147177 May 24 10:37:59.639630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 998/1869] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:37:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:00.656517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:00.657202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:00.663560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:00.664106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:00.664850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:00.665298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:00.797127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:00.805020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-951c54f0-fbba-4467-9f03-f72f09dab949 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152755 May 24 10:38:00.805621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 999/1870] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:00 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:01.822499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:01.823435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:01.830672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:01.831116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:01.831870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:01.832301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:01.958711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:01.960873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:01.967381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6fbed719-c1cd-435c-b613-f1e558de119d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148644 May 24 10:38:01.967947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1000/1871] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:02.983992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:02.984669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:02.991657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:02.992106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:02.992864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:02.993320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:03.126779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:03.128051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0ad133d4-3486-4702-baaf-92d596d3a6fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148151 May 24 10:38:03.128628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1001/1872] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:04.145583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:04.146199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:04.152768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:04.153211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:04.154668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:04.155269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:04.288610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:04.296986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cae44277-7b03-48a0-86e8-6f4127da1c12 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155364 May 24 10:38:04.297592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1002/1873] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:04 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:05.314983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:05.315712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:05.322245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:05.322686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:05.323705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:05.324137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:05.453514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:05.469996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4ac9e7ea-75b3-4d56-97f6-1068b74a286b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.159636 May 24 10:38:05.471325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1003/1874] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:05 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:06.487361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:06.488031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:06.494478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:06.494919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:06.495668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:06.496095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:06.624790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:06.632642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-197f4da4-b4f9-44c8-8eaa-0c5be67fd427 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149260 May 24 10:38:06.633204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1004/1875] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:38:07.650219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:07.650921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:07.657733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:07.658165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:07.658921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:07.659350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:07.788053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:07.796505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7ec24413-16e4-4b33-9b79-a8daf455d585 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150108 May 24 10:38:07.797245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1005/1876] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:07 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:08.815095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:08.815782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:08.821871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:08.822350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:08.823380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:08.823814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:08.962692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:08.970839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7c4fe1a6-3fa3-48ad-b7da-136b06d819b5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.160736 May 24 10:38:08.971422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1006/1877] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:09.988062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:09.988730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:09.995099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:09.995544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:09.996302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:09.996729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:10.127859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:10.145260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ea9168e1-283f-4213-b423-bb62431c6092 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.160996 May 24 10:38:10.145881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1007/1878] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:11.163095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:11.163726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:11.170114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:11.170545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:11.171299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:11.171729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:11.300552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:11.308456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b7bdd1c8-c396-47da-ba55-40da391a4c5f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149332 May 24 10:38:11.309031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1008/1879] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:11 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:12.326974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:12.327601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:12.333989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:12.334429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:12.335180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:12.335620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:12.465941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:12.471337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ce36f72d-70dd-4697-a79e-363ff6c0fe7d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148713 May 24 10:38:12.471908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1009/1880] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:12 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:13.488830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:13.489463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:13.491791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=57,nova_api:DELETE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:13.496635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:13.497225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:13.497884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:13.498320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:13.628736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:13.636650 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4d0c0afa-748b-411f-ae35-d85b1ed4222a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151834 May 24 10:38:13.637223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1010/1881] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:14.653785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:14.654488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:14.661117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:14.661598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:14.662366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:14.662801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:14.788167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:14.796492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-63352bfb-5c12-4fca-80dd-fe80aea03d77 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146066 May 24 10:38:14.797066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1011/1882] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:14 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:15.814350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:15.815019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:15.821443 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:15.821899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:15.822701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:15.823132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:15.949182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:15.958043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0dd74286-a728-4976-9e3a-b934fb182125 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148475 May 24 10:38:15.958617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1012/1883] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:16.974838 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:16.975546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:16.981926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:16.982376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:16.983131 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:16.983566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:17.112370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:17.120173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6480b558-f658-4be0-b1f7-262b62f1f968 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149318 May 24 10:38:17.120740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1013/1884] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:18.136978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:18.137693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:18.145035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:18.145571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:18.146355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:18.146797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:18.285891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:18.293883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ba5b3e07-8c54-4894-bacc-a847b2412caa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.160697 May 24 10:38:18.294495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1014/1885] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:18 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:19.311702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:19.313006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:19.319507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:19.319948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:19.320701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:19.321129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:19.449893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:19.458035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-677bc507-d4df-4470-9304-fc298dcb5594 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150193 May 24 10:38:19.458602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1015/1886] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:19 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:20.475397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:20.476083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:20.482496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:20.482934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:20.483690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:20.484250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:20.612942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:20.620822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d00ff7ae-0e83-4b90-9345-018183f4b255 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149577 May 24 10:38:20.621407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1016/1887] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:20 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:21.638152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:21.638824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:21.646660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:21.647100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:21.647884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:21.648312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:21.663366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:21.778432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:21.786475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c34c9c80-fb61-417c-9ae4-8b00fb8deb74 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152210 May 24 10:38:21.787048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1017/1888] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:21 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:22.803785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:22.804426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:22.810802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:22.811249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:22.812006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:22.812439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:22.938700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:22.946595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0bf9807b-58a6-4a0c-941f-d1024a62847e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146926 May 24 10:38:22.947200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1018/1889] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:23.963998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:23.964703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:23.971452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:23.971897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:23.972648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:23.973077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:24.098713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:24.106572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-575832f0-0839-4e38-813c-2702dbf92eba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146455 May 24 10:38:24.107135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1019/1890] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:23 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:25.123767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:25.124459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:25.131480 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:25.131921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:25.132681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:25.133109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:25.266966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:25.274860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ecd7b650-5314-4114-98f6-74ddbef820d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154917 May 24 10:38:25.275426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1020/1891] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:25 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:26.292418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:26.293062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:26.299632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:26.300072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:26.300821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:26.301302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:26.427553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:26.435375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a8db43cb-d637-4fa0-85ce-c46d83603ee9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146868 May 24 10:38:26.435944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1021/1892] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:26 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:27.454155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:27.454933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:27.461424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:27.461879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:27.462644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:27.463081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:27.593799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:27.602417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aa62dc2d-115d-4b18-956a-84a432cb102a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152830 May 24 10:38:27.602987 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1022/1893] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:27 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:28.619590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:28.620294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:28.627234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:28.627677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:28.628426 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:28.628854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:28.759864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:28.768026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5b9408a0-d51f-41b7-8cbb-1a20b3d3bf4b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152379 May 24 10:38:28.768581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1023/1894] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:28 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:29.784947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:29.785648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:29.792095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:29.792550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:29.793338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:29.793767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:29.917710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:29.926153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ab92e48b-0503-45b3-8456-2f55a10413b1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145118 May 24 10:38:29.926730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1024/1895] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:29 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:30.943058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:30.943758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:30.950397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:30.950876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:30.951634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:30.952066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:31.081722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:31.090652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-68de65b1-bfdc-42bc-9cf6-f6452febfd9b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151687 May 24 10:38:31.091227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1025/1896] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:30 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:32.107714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:32.108401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:32.115371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:32.115820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:32.116600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:32.117018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:32.245399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:32.254019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f93efa32-9392-4ea1-87f5-9c9408723c48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150135 May 24 10:38:32.254593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1026/1897] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:32 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:33.271333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:33.271981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:33.278486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:33.278937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:33.279701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:33.280137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:33.411930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:33.419927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-03bbd6a7-330d-4171-a115-8cd86f97b8a4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152436 May 24 10:38:33.420437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1027/1898] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:33 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:34.436436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:34.440379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:34.444456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:34.444902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:34.445764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:34.446332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:34.571690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:34.580305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4acb71df-8d6d-4ce9-9a18-788ee52ea4d1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147865 May 24 10:38:34.580874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1028/1899] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:34 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:35.597640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:35.598292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:35.604577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:35.605021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:35.605811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:35.606243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:35.736439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:35.744562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6ee4dce6-a15c-4695-836f-32645eca2d87 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150926 May 24 10:38:35.745151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1029/1900] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:35 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:36.762417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:36.763114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:36.770112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:36.770554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:36.771392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:36.771821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:36.897899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:36.905856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-504a1717-caff-4620-b82c-a12ed1248951 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147311 May 24 10:38:36.906441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1030/1901] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:36 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:37.922805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:37.923498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:37.930083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:37.930527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:37.931309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:37.931740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:38.059404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:38.067284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6dcb484b-dcf7-4a98-98df-6d70b6274d92 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148522 May 24 10:38:38.067846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1031/1902] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:37 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:39.084774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:39.085492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:39.091301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:39.091753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:39.092511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:39.092944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:39.223474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:39.231208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-17ce9015-d69b-4faf-8a9f-432ff9d5cf51 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150522 May 24 10:38:39.231770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1032/1903] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:39 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:40.249118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:40.249806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:40.257146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:40.257633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:40.258399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:40.258839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:40.387171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:40.395236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-983499e8-723e-4cf6-84a1-5aa156e7be8d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149890 May 24 10:38:40.406130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1033/1904] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:40 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:41.412498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:41.413125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:41.419579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:41.419965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:41.420716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:41.421143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:41.423868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:41.551440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:41.559347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0bf2a437-895f-4fce-be02-44d844526bf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150703 May 24 10:38:41.559919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1034/1905] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:41 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:42.577288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:42.577953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:42.584920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:42.585415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:42.586187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:42.586624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:42.712858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:42.721343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-06d76dc6-99c9-40d8-b224-345f235e0023 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148149 May 24 10:38:42.721918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1035/1906] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:42 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:43.738437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:43.739060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:43.741310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:43.746065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:43.746518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:43.747281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:43.747720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:43.873585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:43.883182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6b7fe5bf-27be-48e4-9a80-4311e7828dfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148679 May 24 10:38:43.883752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1036/1907] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:43 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:44.900483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:44.901170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:44.908232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:44.908676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:44.909512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:44.909945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:45.040436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:45.048356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f620c1a9-463d-462c-af37-5d24fce7adb2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151970 May 24 10:38:45.048921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1037/1908] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:44 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:46.065943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:46.066601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:46.073004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:46.073491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:46.074509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:46.074943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:46.203160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:46.211930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7aa9d133-5ef8-43c0-aa86-dfb735759ae3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149844 May 24 10:38:46.212529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1038/1909] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:46 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:46.325109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:38:46.325326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:38:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:38:47.229500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:47.230130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:47.236612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:47.237054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:47.237840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:47.238269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:47.367103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:47.374980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0ade6dc4-e6d6-4a35-86dd-e14ceddfa98f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149357 May 24 10:38:47.376037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1039/1910] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:47 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:48.392729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:48.393415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:48.399783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:48.400241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:48.401022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:48.401490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:48.532338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:48.540145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-025f48c7-eb07-4a98-9416-6a8a2a5e1b60 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152178 May 24 10:38:48.540706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1040/1911] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:48 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:49.557045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:49.557742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:49.564734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:49.565177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:49.566036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:49.566465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:49.690753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:49.698858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b5e6773d-0627-434b-9c28-2e0511894f5d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145612 May 24 10:38:49.699720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1041/1912] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:49 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:50.716003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:50.716704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:50.723995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:50.724473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:50.725260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:50.725718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:50.854700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:50.862586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-02ffa82d-a107-487c-ac5d-17d4c61d7a00 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150530 May 24 10:38:50.863152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1042/1913] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:50 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:51.879605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:51.880322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:51.886745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:51.887195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:51.887990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:51.888425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:52.015933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:52.023752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c4a4b913-c373-45bd-890e-4283b8d02073 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148050 May 24 10:38:52.024308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1043/1914] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:51 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:53.041632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:53.042332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:53.049628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:53.050074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:53.050841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:53.051276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:53.187142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:53.195419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-50eb5dcc-74b8-4db3-9cda-fa30ab01a229 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.157814 May 24 10:38:53.196000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1044/1915] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:53 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:54.212871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:54.213558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:54.219999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:54.220442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:54.221335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:54.221769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:54.349882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:54.357751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-502326fe-6798-4a96-8e24-09e62c0fad29 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148690 May 24 10:38:54.358316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1045/1916] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:38:55.375337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:55.376049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:55.382898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:55.383344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:55.384239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:55.384669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:55.518840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:55.526969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e936739b-d041-4156-9189-d78f7a564faa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155512 May 24 10:38:55.527550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1046/1917] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:56.544457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:56.545153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:56.552366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:56.553526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:56.553646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:56.554045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:56.685231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:56.693906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0dd3c489-8933-4f18-8d58-47c7e53ff826 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153281 May 24 10:38:56.694481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1047/1918] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:56 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:57.711754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:57.712435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:57.718937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:57.719385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:57.720402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:57.720867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:58.037932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:58.045797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2fbb2a17-7f9b-4840-af6e-25e111d82489 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.337996 May 24 10:38:58.046369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1048/1919] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:57 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:38:59.063634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:38:59.064626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:38:59.072327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:59.072828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:59.073692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:38:59.074154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:38:59.213710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:38:59.221798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c1604d30-8192-4d69-af3a-35b3703a48b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.161988 May 24 10:38:59.222368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1049/1920] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:38:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:00.239561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:00.240164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:00.246681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:00.247128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:00.247891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:00.248322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:00.372408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:00.374670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:00.381094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-33a28c6e-2492-4f80-92c5-b0a6b94fce1e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146155 May 24 10:39:00.381694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1050/1921] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:00 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:01.398126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:01.398752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:01.405633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:01.406086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:01.406849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:01.407285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:01.536319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:01.544247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-362c9a48-f83c-4f9f-a9c6-b5af5067fc81 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149941 May 24 10:39:01.544934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1051/1922] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:02.561471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:02.562168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:02.568753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:02.569199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:02.569999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:02.570448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:02.696847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:02.704917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2b0af1f0-de10-4e19-9b58-12eae0c518ea tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147323 May 24 10:39:02.706013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1052/1923] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:03.722559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:03.723256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:03.729656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:03.730100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:03.730850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:03.731279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:03.858686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:03.867192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6925fad5-3367-4ce0-ba52-d4d3c522af48 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149044 May 24 10:39:03.867756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1053/1924] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:03 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:39:04.884634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:04.885340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:04.891673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:04.892114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:04.892866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:04.893361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:05.239637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:05.247731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3690ab61-6d1c-41a4-a59e-d6d429d9eecb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.366843 May 24 10:39:05.248303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1054/1925] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:04 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:06.265210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:06.265884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:06.273548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:06.274021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:06.274782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:06.275218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:06.401217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:06.409063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6ecf3cec-ffc1-40f0-8ef5-cd5ac65be368 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147807 May 24 10:39:06.409662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1055/1926] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:07.426765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:07.427380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:07.433795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:07.434244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:07.435006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:07.435441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:07.564349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:07.572617 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d5ed1599-6262-4ab0-ad8e-dab8e74d18c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150409 May 24 10:39:07.573226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1056/1927] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:07 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:08.590892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:08.591592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:08.597989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:08.598427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:08.599169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:08.599595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:08.726558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:08.734550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-211b8431-a50d-447a-9a28-5d8ebae7ec4f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147541 May 24 10:39:08.735121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1057/1928] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:09.751933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:09.752670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:09.759546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:09.759989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:09.760737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:09.761161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:09.890165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:09.899067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7db68112-da20-4814-b666-64f89eba51e5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151015 May 24 10:39:09.899640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1058/1929] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:10.916391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:10.917015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:10.923389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:10.923828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:10.924582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:10.925052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:11.051100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:11.058965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1ecfe51-35f0-43b9-9a53-18688d887272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146431 May 24 10:39:11.059525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1059/1930] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:10 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:12.075740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:12.076358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:12.083070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:12.083522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:12.084288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:12.084728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:12.212012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:12.219986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b25d80ca-e7b8-44a7-944a-ef02886e925a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148449 May 24 10:39:12.220565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1060/1931] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:12 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:13.238008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:13.238718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:13.244978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:13.245460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:13.246216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:13.246648 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:13.376078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:13.384520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-70806e99-ebee-430c-95b5-adb1151b278a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150962 May 24 10:39:13.385110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1061/1932] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:14.401735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:14.402404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:14.404677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:14.410033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:14.416378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:14.416378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:14.416378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:14.538703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:14.546621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d024afcc-0ea7-40a8-9558-33b154628dbd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148726 May 24 10:39:14.547184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1062/1933] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:14 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:15.563915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:15.564582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:15.571665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:15.572114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:15.572872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:15.573325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:15.702483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:15.710814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e25f14f0-255b-4f1f-959a-c9e77f9ae92d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150804 May 24 10:39:15.711384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1063/1934] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:16.727121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:16.727833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:16.734208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:16.734653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:16.735410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:16.735843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:16.860658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:16.869168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb08f1c6-9eef-4af4-a374-3fabfadaf5af tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145934 May 24 10:39:16.869771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1064/1935] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:17.887017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:17.887693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:17.894113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:17.894563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:17.895962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:17.896393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:18.020829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:18.028697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6bf459a1-c6fa-49e8-935c-15f7cd4e64bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145527 May 24 10:39:18.029284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1065/1936] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:17 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:19.045861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:19.046490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:19.053403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:19.053906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:19.054710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:19.055149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:19.187853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:19.195947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c0c2de2b-f343-4fc1-9d13-da7d95886e1a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154006 May 24 10:39:19.196517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1066/1937] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:19 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:20.214201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:20.214823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:20.221227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:20.221698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:20.222447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:20.222873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:20.237640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:20.351395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:20.359367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-35d7a68f-0596-40a7-985f-68b9b5b93ec0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148975 May 24 10:39:20.359938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1067/1938] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:20 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:21.376803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:21.377457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:21.384465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:21.384908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:21.385701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:21.386127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:21.515476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:21.523409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-95b3ab33-fb3d-4b85-abfe-2f4bd519e917 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150471 May 24 10:39:21.523971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1068/1939] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:21 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:22.540635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:22.541332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:22.547923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:22.548372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:22.549126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:22.549731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:22.678939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:22.686904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0bb9ed0c-50ed-4978-b73b-e01f276c3020 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150122 May 24 10:39:22.687481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1069/1940] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:23.705016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:23.705710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:23.712224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:23.712672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:23.713462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:23.713895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:23.843301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:23.851801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-973df487-1e25-42da-8dd9-86ff5555907d tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150679 May 24 10:39:23.852367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1070/1941] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:23 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:24.868922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:24.869592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:24.875967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:24.876411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:24.877167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:24.877620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:25.003798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:25.012307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-17a7e784-e466-4909-a6ce-89631e42d345 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147347 May 24 10:39:25.012882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1071/1942] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:24 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:26.029441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:26.030127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:26.037398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:26.037843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:26.038592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:26.039021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:26.166640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:26.174612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1e85ab43-2072-4cda-9d9a-9adab5a76d7c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149029 May 24 10:39:26.175570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1072/1943] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:26 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:27.191859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:27.192457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:27.198978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:27.199425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:27.200182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:27.200863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:27.331377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:27.339747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3c5f61f3-5f47-48a7-a042-abe653640e40 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151743 May 24 10:39:27.340312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1073/1944] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:27 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:28.356599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:28.357260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:28.363857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:28.364305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:28.365202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:28.365711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:28.490851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:28.499001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d3fa3fa8-f448-4988-91b5-84bfddb45373 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146280 May 24 10:39:28.499566 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1074/1945] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:28 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:29.515860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:29.516500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:29.523778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:29.524220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:29.524969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:29.525445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:29.654751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:29.663354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b0cb451e-fee2-460c-8caa-80555bcca306 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151359 May 24 10:39:29.663917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1075/1946] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:29 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:30.679857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:30.680541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:30.687034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:30.687484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:30.688233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:30.688662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:30.813418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:30.821747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-24e8ff2f-d785-46b7-b106-5cf5ddb1aabc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145743 May 24 10:39:30.822314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1076/1947] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:30 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:31.839568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:31.840196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:31.846690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:31.847133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:31.847882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:31.848311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:31.972529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:31.980990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fb6b2ea2-bb04-444e-9c39-3f5456a2196a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145320 May 24 10:39:31.981590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1077/1948] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:31 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:32.998009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:32.998757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:33.005095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:33.005599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:33.006906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:33.007385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:33.140472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:33.149317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cbb786e7-5749-4401-b886-ea9ac652b2a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155191 May 24 10:39:33.149897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1078/1949] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:32 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:34.167153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:34.167818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:34.174506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:34.174944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:34.175697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:34.176127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:34.307822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:34.316056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-da23bc05-0eb4-4baf-8910-8673d5016813 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153534 May 24 10:39:34.316625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1079/1950] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:34 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:35.334281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:35.335512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:35.342115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:35.342776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:35.343802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:35.344310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:35.639109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:35.646881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-65051381-fa9d-4910-8c74-193d5f6ae2fd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.316614 May 24 10:39:35.648070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1080/1951] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:35 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:36.663938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:36.664653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:36.671232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:36.671737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:36.672495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:36.672930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:36.799281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:36.808082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-04bc298e-9f63-4ecc-ae6c-8ed0243ab7d3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148000 May 24 10:39:36.808639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1081/1952] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:36 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:37.825149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:37.826097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:37.832403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:37.832847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:37.833657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:37.834087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:37.961839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:37.969644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5d7a0dbf-f2a0-4d0f-a468-5d002543bb42 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148260 May 24 10:39:37.970225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1082/1953] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:37 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:38.985937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:38.986618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:38.992992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:38.993486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:38.994254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:38.994715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:39.120517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:39.128610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-11078d11-0443-4f46-9ba1-81a0bfb0f80a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146586 May 24 10:39:39.129164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1083/1954] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:38 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:40.145814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:40.146488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:40.153042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:40.153526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:40.154324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:40.154758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:40.158644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:40.282339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:40.291240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4c557ced-1de2-4c8a-9c17-90888e55b374 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149307 May 24 10:39:40.291797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1084/1955] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:40 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:41.308704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:41.309361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:41.315753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:41.316194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:41.316950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:41.317406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:41.447421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:41.455524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-08678910-f419-44c0-aadd-2c1c5d1b587a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150665 May 24 10:39:41.456094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1085/1956] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:41 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:42.473143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:42.473832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:42.480404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:42.480853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:42.481643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:42.482074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:42.609642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:42.617991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f1c53f3a-09d0-4963-bcaa-27ecfe46bfec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148712 May 24 10:39:42.618526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1086/1957] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:42 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:43.635345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:43.636017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:43.642512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:43.642964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:43.643726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:43.644165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:43.775704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:43.783666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-39ac2951-f318-4686-9573-6b1ac5699bbc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152385 May 24 10:39:43.784559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1087/1958] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:43 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:44.801124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:44.801824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:44.804678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:44.809406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:44.809899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:44.810652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:44.811079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:44.936227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:44.944174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d07b2ea1-5b97-4acb-a1fa-59b4876e5642 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146946 May 24 10:39:44.944764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1088/1959] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:44 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:45.961890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:45.962546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:45.969376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:45.969904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:45.970687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:45.971129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:46.102598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:46.112473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9759f49c-90c1-4f99-9b2e-694f0cebe6ec tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153006 May 24 10:39:46.113252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1089/1960] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:45 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:47.130247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:47.130932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:47.138208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:47.138666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:47.139429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:47.139862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:47.269632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:47.277860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a6af933-26eb-472b-a735-50ab89717c3c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151467 May 24 10:39:47.278434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1090/1961] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:47 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:48.294817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:48.295445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:48.301895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:48.302338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:48.303085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:48.303738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:48.429171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:48.437147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cc5dbb67-47d2-4d50-98b8-ac84d8c2fd82 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146104 May 24 10:39:48.437741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1091/1962] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:48 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:49.455223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:49.455857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:49.463056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:49.463499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:49.464245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:49.464672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:49.596220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:49.604916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b3c74cc9-0849-46e8-a7df-7d5951541800 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153785 May 24 10:39:49.605509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1092/1963] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:49 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:50.622672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:50.623388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:50.629858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:50.630301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:50.631050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:50.631720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:50.761983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:50.770003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c3ebfbb8-9128-4609-bb5f-48fab4313164 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151178 May 24 10:39:50.770577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1093/1964] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:50 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:39:51.786998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:51.787727 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:51.794147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:51.794591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:51.795344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:51.795776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:51.921119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:51.929866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dd468b04-1466-4865-b30b-28b9db3a612c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146729 May 24 10:39:51.930437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1094/1965] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:51 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:52.947513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:52.948223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:52.954870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:52.955316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:52.956068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:52.956499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:53.085353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:53.093163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0a11f609-2118-4dbc-95fb-3700bfdf961c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150151 May 24 10:39:53.093759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1095/1966] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:52 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:54.110708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:54.111346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:54.118075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:54.118525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:54.119287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:54.119734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:54.249902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:54.258459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ba73a943-cbe0-4e8a-977a-9e66887fed14 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151724 May 24 10:39:54.259060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1096/1967] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:55.275673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:55.276354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:55.282786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:55.283293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:55.284082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:55.284516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:55.414813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:55.422843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c56649be-d821-4cde-ba45-cc6baf3b3084 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151038 May 24 10:39:55.423348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1097/1968] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:56.440065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:56.440746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:56.448117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:56.448565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:56.449364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:56.449805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:56.603286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:56.611411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-79e690f4-f6ba-4d3b-9076-ca1426888034 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.175290 May 24 10:39:56.611976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1098/1969] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:56 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:57.627972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:57.628653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:57.635778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:57.636220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:57.636968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:57.637424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:57.764483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:57.772349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-219b82eb-12e6-4086-8711-c3c927f927bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148224 May 24 10:39:57.773438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1099/1970] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:57 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:58.791340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:58.792022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:58.798442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:58.798889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:58.799642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:58.800075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:58.930788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:39:58.937257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:58.939412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-32534ff8-506b-496d-9dba-11a7f6d8c9aa tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152467 May 24 10:39:58.939978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1100/1971] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:58 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:39:59.957387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:39:59.958066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:39:59.964736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:59.965180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:39:59.966077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:39:59.966505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:00.090886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:00.098959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-34ada927-d6b3-4b45-b32c-ef92608c463f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145725 May 24 10:40:00.099530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1101/1972] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:39:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:01.116450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:01.117192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:01.123798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:01.124243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:01.124998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:01.125465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:01.254304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:01.262829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b5602c1a-5476-4d1b-b1a1-9a886f16cdcb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150281 May 24 10:40:01.263394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1102/1973] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:02.279827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:02.280544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:02.287004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:02.287454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:02.288212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:02.289471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:02.414925 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:02.422698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-02c68a02-4368-4103-974b-e047c2511678 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146734 May 24 10:40:02.423255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1103/1974] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:03.439705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:03.440336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:03.446920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:03.447363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:03.448112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:03.448542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:03.578163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:03.586666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0041d24b-bc6c-4637-81dc-95d8bd5e80f8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150770 May 24 10:40:03.587230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1104/1975] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:03 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:04.603828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:04.604502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:04.611150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:04.611596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:04.612353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:04.612783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:04.747350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:04.755181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0e0812db-ed94-4767-9ad2-c121b72a60e8 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155307 May 24 10:40:04.755743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1105/1976] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:04 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:05.771742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:05.772422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:05.779518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:05.779961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:05.780716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:05.781147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:05.906879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:05.915184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1509594f-d7d5-419d-95d4-49964ff80201 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147439 May 24 10:40:05.915744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1106/1977] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:05 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:06.932302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:06.932952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:06.939381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:06.939831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:06.940587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:06.941023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:07.070857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:07.078956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c411cf46-978f-41b3-bbdf-089383d8c869 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151259 May 24 10:40:07.079515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1107/1978] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:06 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:08.095920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:08.096601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:08.102993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:08.103439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:08.104193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:08.104621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:08.230287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:08.238048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0c211456-246c-4fe3-8dc2-a714ca0ece08 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145991 May 24 10:40:08.238607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1108/1979] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:08 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:09.256074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:09.256724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:09.263061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:09.263512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:09.264270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:09.264704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:09.394466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:09.402389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f44043b1-4c4e-45f7-8119-712642b5b08f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150365 May 24 10:40:09.402958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1109/1980] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:09 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:10.420904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:10.420904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:10.426646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:10.427095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:10.427850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:10.428283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:10.560250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:10.568287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef2a8f8c-18b0-419e-8d0a-065431d50272 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152616 May 24 10:40:10.568856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1110/1981] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:10 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:11.585770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:11.586622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:11.593487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:11.593937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:11.594695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:11.595135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:11.719690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:11.727655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-69d51ab9-fade-4e38-984e-f97462675c6f tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145734 May 24 10:40:11.728220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1111/1982] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:11 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:12.745549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:12.746293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:12.752618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:12.753068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:12.753862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:12.754295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:12.885788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:12.893572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6a0d4447-9123-44e1-b835-4ab5e7b7e51b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152571 May 24 10:40:12.894138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1112/1983] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:12 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:13.910740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:13.911371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:13.918525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:13.918970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:13.919724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:13.920150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:14.045598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:14.053404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c1d963cd-b186-4270-9dfb-9cd5f294f875 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146622 May 24 10:40:14.053962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1113/1984] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:13 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:15.070662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:15.072071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:15.074292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:40:15.079256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:15.079701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:15.080507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:15.080934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:15.214073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:15.222162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6c4ce0e9-902c-4b22-b409-e9ee2c5dc4bd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.155310 May 24 10:40:15.222737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1114/1985] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:15 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:16.239707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:16.240387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:16.247134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:16.247576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:16.248322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:16.248752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:16.376737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:16.385437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-411f436b-da3d-4788-b10f-92c368f0a145 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149685 May 24 10:40:16.386009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1115/1986] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:16 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:17.402970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:17.403600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:17.410084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:17.410527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:17.411283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:17.411711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:17.541178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:17.549237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4926d042-7385-468d-9459-a21a924699b7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150975 May 24 10:40:17.549952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1116/1987] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:17 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:18.566712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:18.567383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:18.573915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:18.574365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:18.575146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:18.575573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:18.590632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:40:18.704290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:18.712699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-77491937-3c15-42f6-a36c-36ebe266756e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149787 May 24 10:40:18.719391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1117/1988] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:18 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:19.730790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:19.731437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:19.737928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:19.738375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:19.739132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:19.739565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:19.864402 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:19.872947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ac547c58-b47f-49cb-af46-43060dca2942 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146531 May 24 10:40:19.873550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1118/1989] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:19 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:20.890516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:20.891192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:20.897652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:20.898114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:20.898875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:20.900001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:21.030518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:21.039685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a23e31e9-1b35-4e1d-9d16-f2444e3f9021 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153034 May 24 10:40:21.040270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1119/1990] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:20 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:22.056229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:22.056914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:22.063273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:22.063729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:22.064523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:22.064950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:22.194440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:22.202639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9a12e180-a068-4611-99a3-4cdb68541937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150602 May 24 10:40:22.203211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1120/1991] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:22 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:23.219867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:23.220468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:23.226916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:23.227989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:23.228880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:23.229328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:23.360554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:23.369168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9e6d4a1c-a44b-41e0-8be0-53efa3c14fbb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153080 May 24 10:40:23.369760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1121/1992] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:23 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:24.387139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:24.387826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:24.394249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:24.394704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:24.395464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:24.395901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:24.520872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:24.529014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-04359fa6-9c12-4f7b-8224-43ac2e6be12c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146637 May 24 10:40:24.529665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1122/1993] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:24 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:25.546160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:25.546852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:25.553432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:25.553885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:25.554649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:25.555088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:25.681039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:25.689060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ed6b487c-af5d-413e-b4b7-2bea90164d0a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146776 May 24 10:40:25.689761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1123/1994] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:25 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:26.707221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:26.707917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:26.714805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:26.715256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:26.716026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:26.716466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:26.844928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:26.853508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6e5ce621-fce7-4c61-9aef-dc3c93770126 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150072 May 24 10:40:26.854085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1124/1995] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:26 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:27.871122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:27.871802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:27.878232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:27.878678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:27.879435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:27.879867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:28.006858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:28.016059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8e083618-4300-44c3-a32d-11159fd80235 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149411 May 24 10:40:28.016640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1125/1996] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:27 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:29.033994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:29.034730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:29.041612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:29.042066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:29.042922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:29.043362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:29.174913 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:29.183645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8cf68567-ee0c-431c-9311-24ecea9075ba tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.153588 May 24 10:40:29.184202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1126/1997] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:29 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:29.924815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:40:29.925062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:40:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:40:30.200591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:30.201187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:30.207945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:30.208383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:30.209129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:30.209591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:30.336371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:30.344806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6a783f6e-db40-49d5-af66-39c8500d0fd4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148586 May 24 10:40:30.345389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1127/1998] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:30 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:31.362280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:31.362895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:31.369627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:31.370064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:31.370814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:31.371243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:31.500688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:31.508866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-25d8225f-c568-4f92-8710-4ad40ea643bc tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150274 May 24 10:40:31.509629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1128/1999] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:31 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:40:32.526376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:32.527032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:32.533534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:32.533985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:32.534738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:32.535167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:32.661910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:32.669908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5f10e743-5ffb-4af7-b601-a95b037e768e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147443 May 24 10:40:32.670478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1129/2000] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:32 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:33.687510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:33.688191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:33.694685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:33.695136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:33.695900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:33.696396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:33.822016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:33.830208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fe3b947e-c7b8-4e7d-a0bc-4b0a2c9d79cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146543 May 24 10:40:33.830777 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1130/2001] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:33 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:34.847609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:34.848324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:34.854870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:34.855317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:34.856084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:34.856516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:34.987157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:34.995539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8f096506-66eb-4da7-828a-8bab42d1a1a7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151953 May 24 10:40:34.996109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1131/2002] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:34 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:36.012527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:36.013249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:36.019610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:36.020051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:36.020797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:36.021233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:36.152773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:36.160843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e262cdae-12db-41b2-871b-ffc579c3d3cb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152793 May 24 10:40:36.161434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1132/2003] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:36 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:37.179229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:37.179858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:37.186725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:37.187176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:37.187937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:37.188368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:37.319716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:37.329122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-09c5e485-dfb3-4673-a81a-1cdd0e761571 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.154854 May 24 10:40:37.329714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1133/2004] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:37 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:37.405165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:40:37.405358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:40:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:40:38.348653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:38.349377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:38.356429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:38.356872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:38.357654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:38.358086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:38.360559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:40:38.485506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:38.493366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9019a35b-980e-4527-9f17-8c30d4f430fb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.148541 May 24 10:40:38.493955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1134/2005] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:38 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:39.510979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:39.511749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:39.517916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:39.518370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:39.519141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:39.519580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:39.648900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:39.657006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8de442dd-5cfc-4e9a-af79-ab37709ec965 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.150755 May 24 10:40:39.657606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1135/2006] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:39 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:40.674921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:40.675558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:40.681932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:40.682380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:40.683138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:40.683569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:40.814135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:40.822066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-018b878a-80a0-416a-ba4e-c6cc98d488f1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151348 May 24 10:40:40.825985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1136/2007] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:40 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:41.839961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:41.840751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:41.847519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:41.847963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:41.848709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:41.849139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:41.974601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:41.983015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2102f01e-1172-44e1-862c-d439bc37fcf2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146835 May 24 10:40:41.983521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1137/2008] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:41 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:43.000723 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:43.001500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:43.007866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:43.008314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:43.009068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:43.009527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:43.137794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:43.145879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-801b0e42-acc3-4bc8-a60a-e4389bdc3cc4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149636 May 24 10:40:43.153144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1138/2009] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:42 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:44.168125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:44.168787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:44.175306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:44.175765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:44.176520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:44.176953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:44.305658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:44.313818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-94c6e735-5c77-41b2-86da-a4b92fdbb89c tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149765 May 24 10:40:44.314395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1139/2010] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:44 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:45.332031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:45.332644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:45.334985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=52 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:40:45.339781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:45.340259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:45.341032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:45.341515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:45.472612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:45.481299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0fb02a1f-609b-4bd3-9a64-539b89ec20d2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152946 May 24 10:40:45.481882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1140/2011] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:45 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:46.498532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:46.499208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:46.505700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:46.506146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:46.506897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:46.507323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:46.634681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:46.643644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f63c9e1a-81da-4214-b691-8ef25e689277 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149069 May 24 10:40:46.644260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1141/2012] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:46 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:47.660547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:47.661219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:47.667451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:47.667894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:47.669348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:47.669786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:47.796524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:47.804495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1b2abfd7-d138-4b05-882a-186b76224c52 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147803 May 24 10:40:47.805058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1142/2013] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:47 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:48.821657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:48.822325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:48.829671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:48.830114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:48.830863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:48.831292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:48.961853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:48.969804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-14ae3333-1c77-4fe1-ba69-8a942c7854a0 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152008 May 24 10:40:48.972977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1143/2014] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:48 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:49.989171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:49.989892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:49.996839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:49.997379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:49.998144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:49.998579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:50.124546 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:50.132330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6e7d7e00-a760-4539-ba6c-fdccb6821dee tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146986 May 24 10:40:50.132893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1144/2015] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:49 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:51.150205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:51.150883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:51.157628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:51.158078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:51.158836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:51.159275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:51.290564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:51.299076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d5b45938-ce8b-4f05-86f1-6d52d2dc36c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.152815 May 24 10:40:51.299639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1145/2016] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:51 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:52.316106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:52.316757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:52.323192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:52.323642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:52.324484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:52.325032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:52.450145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:52.458424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-50971372-7ae0-4fc6-9504-b92be8791b21 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.146139 May 24 10:40:52.458997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1146/2017] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:52 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:53.475653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:53.476292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:53.482864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:53.483317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:53.484102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:53.484535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:53.609229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:53.617117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bb27d2b0-34af-45f5-aa33-0a249e1837c1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.145314 May 24 10:40:53.617717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1147/2018] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:53 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:54.634645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:54.635254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:54.642454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:54.642899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:54.643659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:54.644098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:54.772132 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:54.780611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec1ac4d1-7987-46e5-9fe7-0e968934c0c2 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.149785 May 24 10:40:54.781170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1148/2019] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:54 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:55.796974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:55.797689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:55.804073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:55.804513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:55.805261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:55.805709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:55.932271 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:55.940608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d772752e-dbe4-4b84-b421-c010afbfdfa5 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.147660 May 24 10:40:55.941166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1149/2020] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:55 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:56.959033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:56.959677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:56.966758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:56.967261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:56.968021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:56.968456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:57.326219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:57.328656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:40:57.339001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e37aae6e-c161-4ade-9356-422667ff1a64 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.380036 May 24 10:40:57.339001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1150/2021] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:56 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:58.469973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:58.470590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:58.476586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:58.477036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:58.477824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:58.478266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:58.605068 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:58.613254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-48ac0b55-2087-47fa-a28d-60bfc3027432 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.265476 May 24 10:40:58.613865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1151/2022] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:58 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:40:59.631067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:40:59.631812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:40:59.638870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:59.639317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:59.640082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:40:59.640544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:40:59.770389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:40:59.779025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-429f3f9b-47e0-4fad-8abb-cea658c0cb36 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1559 microversion: 2.1 time: 0.151962 May 24 10:40:59.779585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1152/2023] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:40:59 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:41:00.458017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-8d296174-f7ca-41c0-b35c-1c754f94d18c req-9a8925cd-23ce-47d1-96e5-5228b97243a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4942f39-67fa-4c51-b915-7354d2c78f00", "name": "network-vif-deleted", "tag": "55b9120e-64fd-43d8-bfff-e89f1183b17a"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:41:00.466965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-8d296174-f7ca-41c0-b35c-1c754f94d18c req-9a8925cd-23ce-47d1-96e5-5228b97243a4 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:00.469899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-8d296174-f7ca-41c0-b35c-1c754f94d18c req-9a8925cd-23ce-47d1-96e5-5228b97243a4 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:00.478704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-8d296174-f7ca-41c0-b35c-1c754f94d18c req-9a8925cd-23ce-47d1-96e5-5228b97243a4 service nova] Creating event network-vif-deleted:55b9120e-64fd-43d8-bfff-e89f1183b17a for instance c4942f39-67fa-4c51-b915-7354d2c78f00 on hv-cinder-84311 May 24 10:41:00.485619 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-8d296174-f7ca-41c0-b35c-1c754f94d18c req-9a8925cd-23ce-47d1-96e5-5228b97243a4 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091162 May 24 10:41:00.486217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1153/2024] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:41:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:41:00.796944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:41:00.797655 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:41:00.803904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:00.804295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:00.805054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:00.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:00.866525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:41:00.875257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4b801bf0-42db-4b7b-9486-3ea166d9011e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1349 microversion: 2.1 time: 0.083027 May 24 10:41:00.875819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1154/2025] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:41:00 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1349 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:41:01.892071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:41:01.892873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:41:01.899242 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:01.899684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:01.900446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:01.900875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:01.963423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:41:01.971672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-66c5461f-6f63-488e-833c-83f8764ba71e tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1349 microversion: 2.1 time: 0.083610 May 24 10:41:01.972238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1155/2026] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:41:01 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1349 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:41:02.988550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:41:02.989200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: c4942f39-67fa-4c51-b915-7354d2c78f00] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:41:02.995623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:02.996080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:02.996842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:41:02.997299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:41:03.060649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:41:03.069538 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-22a48b76-6f40-4032-aaa7-4d1165ce5db9 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00" status: 200 len: 1349 microversion: 2.1 time: 0.084765 May 24 10:41:03.070107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1156/2027] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:41:02 2022] GET /compute/v2.1/servers/c4942f39-67fa-4c51-b915-7354d2c78f00 => generated 1349 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:42:06.334153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:42:06.337606 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:42:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:43:03.191776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=33 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:03.192539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=33 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:03.197282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:43:03.197774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:43:03.203563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:43:03.204008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:43:03.204766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:43:03.205197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:43:03.219259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] HTTP exception thrown: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Request method failure captured: May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 HTTP/1.1 May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_TJiJC50rIjdcS63u7xuYhM4UlnZwxRv8mUKp0BzcDusYujaMOjfiQ7B-7czQB8mmEz41Y8c7oBuVaj_T84-448VkgY0Pf5gNrx-P1niD879m3w66ryHsb9xDN21s3gpWLUkY4irhyPqPHyYMccDT3Xqb0fUHM9qSZRWOI7k2kkeY May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:43:03.220954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeTestJSON-1912151979 May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 7bffd469507b4dbc8d3146f051c8c8ad May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.222704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '4ca33d7e-37bd-4d91-8a41-866155df1bc3'} May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:43:03.223565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.224939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.226666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Returning 404 to user: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:43:03.226666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d9b4d49b-a9bc-44fe-9329-673ca500e3b3 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 404 len: 111 microversion: 2.1 time: 0.032741 May 24 10:43:03.226666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1157/2028] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:43:03 2022] DELETE /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:43:03.230490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:43:03.231092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] [instance: 4ca33d7e-37bd-4d91-8a41-866155df1bc3] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:43:03.236877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:43:03.237348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:43:03.238125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:43:03.238556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:43:03.252062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] HTTP exception thrown: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Request method failure captured: May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 HTTP/1.1 May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_TJiJC50rIjdcS63u7xuYhM4UlnZwxRv8mUKp0BzcDusYujaMOjfiQ7B-7czQB8mmEz41Y8c7oBuVaj_T84-448VkgY0Pf5gNrx-P1niD879m3w66ryHsb9xDN21s3gpWLUkY4irhyPqPHyYMccDT3Xqb0fUHM9qSZRWOI7k2kkeY May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:43:03.254409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/a02e2e12ebd14e5d8e960a65f41bfb00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeTestJSON-1912151979 May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: a02e2e12ebd14e5d8e960a65f41bfb00 May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1912151979 May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 7bffd469507b4dbc8d3146f051c8c8ad May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeTestJSON-1912151979-project May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.260758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '4ca33d7e-37bd-4d91-8a41-866155df1bc3'} May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:43:03.261792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. May 24 10:43:03.263441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:43:03.265696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] Returning 404 to user: Instance 4ca33d7e-37bd-4d91-8a41-866155df1bc3 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:43:03.265696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-910866f3-ab40-4383-9bc2-8658d56b41e7 tempest-AttachVolumeTestJSON-1912151979 tempest-AttachVolumeTestJSON-1912151979-project] 10.222.0.35 "GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3" status: 404 len: 111 microversion: 2.1 time: 0.028898 May 24 10:43:03.265696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1158/2029] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:43:03 2022] GET /compute/v2.1/servers/4ca33d7e-37bd-4d91-8a41-866155df1bc3 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:43:49.933490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:43:49.934102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:43:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:43:57.412965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:43:57.413652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:43:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:45:26.342418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:45:26.343047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:45:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:47:09.943803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:47:09.944489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:47:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:47:17.421284 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 10:47:17.421874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 10:47:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 10:48:03.680699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR oslo.messaging._drivers.impl_rabbit [-] [0fc8a675-2bc6-487b-9cd5-d58be25af450] AMQP server on 10.222.0.35:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: May 24 10:48:03.684736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1476246717", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "networks": [{"uuid": "0ecf4be9-cd66-415d-a00c-6313ce09660a"}]}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:03.918498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] validate_networks() for [('0ecf4be9-cd66-415d-a00c-6313ce09660a', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:48:04.325100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:48:04.325549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:48:04.326019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:48:04.326423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:48:04.326833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:48:04.336295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:04.336743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:04.337565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:04.337992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:04.356787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Getting quotas for project 0f0441599dc5456da3e0f48be7dd2d20. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:48:04.360818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Getting quotas for user bb2d87f281834e4697a8c3559b6e83a2 and project 0f0441599dc5456da3e0f48be7dd2d20. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:48:04.373618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:48:04.375906 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:48:04.493941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7155b3ef-1275-4506-bd2e-0a6a08be3388 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.816535 May 24 10:48:04.494589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1159/2030] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:48:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 818 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 24 10:48:04.504095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b2979a85-0470-4057-bdc4-190504875531 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:04.504737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b2979a85-0470-4057-bdc4-190504875531 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:04.565166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b2979a85-0470-4057-bdc4-190504875531 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:04.576519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b2979a85-0470-4057-bdc4-190504875531 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1219 microversion: 2.1 time: 0.074749 May 24 10:48:04.576519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1160/2031] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:04 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1219 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:04.654038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:04.654766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:04.661707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:04.664114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:04.665107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:04.665692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:04.699730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:48:04.706646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] [0fc8a675-2bc6-487b-9cd5-d58be25af450] Reconnected to AMQP server on 10.222.0.35:5672 via [amqp] client with port 50610. May 24 10:48:04.972218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7881bef7-2e76-4c8c-a775-653a815d190a tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "DELETE /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 204 len: 0 microversion: 2.1 time: 0.321689 May 24 10:48:04.977182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1161/2032] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:48:04 2022] DELETE /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:48:04.984690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:04.985568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:04.992356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:04.993060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:04.994576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:04.995183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:05.358134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:05.366003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-18c533b0-3aa6-4b5c-8a84-794f2074b408 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1508 microversion: 2.1 time: 0.387016 May 24 10:48:05.366579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1162/2033] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:04 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1508 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:05.593873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:05.594485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:05.601517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:05.601971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:05.602741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:05.603240 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:05.671186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:05.679338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f26c5358-62f2-46bc-b3bf-b65a9f7660ed tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1223 microversion: 2.1 time: 0.089501 May 24 10:48:05.679691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1163/2034] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:05 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1223 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:06.204260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-aee130c5-23ee-49d3-abbd-5407e46215f4 req-f0c873c3-a3f6-44cf-966b-19dd778b71ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "75b93292-1e43-43c3-913b-eee6a6623e73", "name": "network-vif-deleted", "tag": "da6486e7-8d1e-46c5-abb7-88003d512083"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:06.214670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-aee130c5-23ee-49d3-abbd-5407e46215f4 req-f0c873c3-a3f6-44cf-966b-19dd778b71ce service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:06.214670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-aee130c5-23ee-49d3-abbd-5407e46215f4 req-f0c873c3-a3f6-44cf-966b-19dd778b71ce service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:06.226088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-aee130c5-23ee-49d3-abbd-5407e46215f4 req-f0c873c3-a3f6-44cf-966b-19dd778b71ce service nova] Creating event network-vif-deleted:da6486e7-8d1e-46c5-abb7-88003d512083 for instance 75b93292-1e43-43c3-913b-eee6a6623e73 on hv-cinder-84311 May 24 10:48:06.235936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-aee130c5-23ee-49d3-abbd-5407e46215f4 req-f0c873c3-a3f6-44cf-966b-19dd778b71ce service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108587 May 24 10:48:06.236536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1164/2035] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:48:06.386636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:06.387517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:06.394372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:06.395037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:06.396428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:06.396926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:06.467063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:06.475265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a44f5ab2-b361-4cf0-8fa6-66447f82f35d tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 200 len: 1466 microversion: 2.1 time: 0.093476 May 24 10:48:06.476107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1165/2036] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:06 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 1466 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:06.698978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:06.699589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:06.707267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:06.708683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:06.709686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:06.710336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:06.788903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:06.799358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1dbaa16c-f56f-4e1a-b51d-83fbee33d2e7 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1279 microversion: 2.1 time: 0.107066 May 24 10:48:06.799949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1166/2037] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:06 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1279 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:07.493984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:07.494623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: 75b93292-1e43-43c3-913b-eee6a6623e73] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:07.502919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:07.503526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:07.505094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:07.505587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:07.520625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] HTTP exception thrown: Instance 75b93292-1e43-43c3-913b-eee6a6623e73 could not be found. May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Request method failure captured: May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 HTTP/1.1 May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_Tzo8x-h618vQIK2AZLsdhbocu8prJbiBdNbF4qbpYG1T25CEKu0iXYE0uhR0ccC8E3yZqmtj1CZrNPnmvDbAL3DpOJ2ErogKclQ3hggHrFsUzfMzEWXvOyC-9cvzh23HWiH6chQJDjdzC48vA8e-TI3CS-ooI8UxDpF3wTWTNLUA May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:48:07.522710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: baac9b5e923747e6986720e3ac849056 May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 75b93292-1e43-43c3-913b-eee6a6623e73 could not be found. May 24 10:48:07.524408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '75b93292-1e43-43c3-913b-eee6a6623e73'} May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 75b93292-1e43-43c3-913b-eee6a6623e73 could not be found. May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:48:07.525397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 75b93292-1e43-43c3-913b-eee6a6623e73 could not be found. May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 75b93292-1e43-43c3-913b-eee6a6623e73 could not be found. May 24 10:48:07.527095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:07.530008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Returning 404 to user: Instance 75b93292-1e43-43c3-913b-eee6a6623e73 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:48:07.530008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6de9bac7-1af5-46c3-be00-275793469676 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73" status: 404 len: 111 microversion: 2.1 time: 0.034208 May 24 10:48:07.530008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1167/2038] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:07 2022] GET /compute/v2.1/servers/75b93292-1e43-43c3-913b-eee6a6623e73 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:48:07.532468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:07.532960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:07.539726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:07.540173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:07.540945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:07.541416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:07.573589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:48:07.857008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d8222f88-dfc8-4221-b7eb-9333a6be2d25 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "DELETE /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 204 len: 0 microversion: 2.1 time: 0.327938 May 24 10:48:07.857645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1168/2039] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:48:07 2022] DELETE /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:48:07.863074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:07.863805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:07.870165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:07.870784 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:07.871610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:07.872046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:08.045302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:08.053492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c7293773-1615-49a9-8149-f23e86bc6bcd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.194550 May 24 10:48:08.058019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1169/2040] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:07 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:08.059151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:08.059762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:08.067331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:08.067790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:08.068560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:08.069198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:08.268962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:08.277564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a3126ee-1546-4fb4-8e21-48cd3e949bac tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1523 microversion: 2.1 time: 0.222371 May 24 10:48:08.278177 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1170/2041] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:08 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1523 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:08.414604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-24898723-60ff-49a0-b509-f767f8dd6a98 req-dd0f785e-606d-44eb-9bc5-816a7e5c2b52 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4cd27eab-afa0-46ce-810a-1985984a6034", "tag": "d81c2027-2c87-4fa4-9867-00e0f0f82878"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:08.422677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-24898723-60ff-49a0-b509-f767f8dd6a98 req-dd0f785e-606d-44eb-9bc5-816a7e5c2b52 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:08.424338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-24898723-60ff-49a0-b509-f767f8dd6a98 req-dd0f785e-606d-44eb-9bc5-816a7e5c2b52 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:08.435307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-24898723-60ff-49a0-b509-f767f8dd6a98 req-dd0f785e-606d-44eb-9bc5-816a7e5c2b52 service nova] Creating event network-changed:d81c2027-2c87-4fa4-9867-00e0f0f82878 for instance 4cd27eab-afa0-46ce-810a-1985984a6034 on hv-cinder-84311 May 24 10:48:08.440018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-24898723-60ff-49a0-b509-f767f8dd6a98 req-dd0f785e-606d-44eb-9bc5-816a7e5c2b52 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029929 May 24 10:48:08.440601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1171/2042] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:48:09.073352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:09.074270 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:09.080820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:09.083438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:09.083438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:09.083438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:09.222897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:09.231265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0936eadd-7562-4488-8df9-be7f10645f86 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.162791 May 24 10:48:09.231776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1172/2043] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:09 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:09.295104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:09.295720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:09.302062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:09.302479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:09.303880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:09.304328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:09.337117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:09.337917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:09.338629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lazy-loading 'fault' on Instance uuid b3484367-cf43-4128-b7f2-96124543234c {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:48:09.380303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:09.388058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0b5ce25-a70a-430f-96d5-03108877e4f2 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1312 microversion: 2.1 time: 0.096723 May 24 10:48:09.388647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1173/2044] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:09 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1312 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:10.247882 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:10.248533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:10.255585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:10.256056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:10.256794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:10.257225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:10.387608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:10.391181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:10.396465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e33894f9-4541-48b0-8379-bbb3cf142937 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.152504 May 24 10:48:10.397026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1174/2045] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:10 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:10.404529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:10.405138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:10.411616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:10.412442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:10.413401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:10.413850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:10.446143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:10.446600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:10.447173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lazy-loading 'fault' on Instance uuid b3484367-cf43-4128-b7f2-96124543234c {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:48:10.492303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'baac9b5e923747e6986720e3ac849056', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'af776ddc4c0f4e6390301a02c6bbfca0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:10.499603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-32010ed3-2a63-4987-9157-1d52787ce86b tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 200 len: 1312 microversion: 2.1 time: 0.098732 May 24 10:48:10.500165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1175/2046] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:10 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 1312 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:10.505207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-11465f03-45f7-4e5a-a015-feb54444206d req-6f381516-6dbb-401c-b708-6b445bcb3a99 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b3484367-cf43-4128-b7f2-96124543234c", "name": "network-vif-deleted", "tag": "0241cf1e-f6a3-46ac-b0d9-a787b63f4fcf"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:10.513170 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-11465f03-45f7-4e5a-a015-feb54444206d req-6f381516-6dbb-401c-b708-6b445bcb3a99 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:10.515562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-11465f03-45f7-4e5a-a015-feb54444206d req-6f381516-6dbb-401c-b708-6b445bcb3a99 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:10.526570 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-11465f03-45f7-4e5a-a015-feb54444206d req-6f381516-6dbb-401c-b708-6b445bcb3a99 service nova] Creating event network-vif-deleted:0241cf1e-f6a3-46ac-b0d9-a787b63f4fcf for instance b3484367-cf43-4128-b7f2-96124543234c on hv-cinder-84311 May 24 10:48:10.531496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-11465f03-45f7-4e5a-a015-feb54444206d req-6f381516-6dbb-401c-b708-6b445bcb3a99 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030248 May 24 10:48:10.532055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1176/2047] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:48:11.414604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:11.415315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:11.422016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:11.422811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:11.423678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:11.424124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:11.561219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:11.570497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f54ede8e-702a-4e4d-915c-c91d677940a1 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.161032 May 24 10:48:11.571084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1177/2048] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:11 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:11.575517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:11.578958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] [instance: b3484367-cf43-4128-b7f2-96124543234c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:11.582506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:11.582963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:11.583730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:11.584196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:11.598321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] HTTP exception thrown: Instance b3484367-cf43-4128-b7f2-96124543234c could not be found. May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Request method failure captured: May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c HTTP/1.1 May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijK_Tzo8x-h618vQIK2AZLsdhbocu8prJbiBdNbF4qbpYG1T25CEKu0iXYE0uhR0ccC8E3yZqmtj1CZrNPnmvDbAL3DpOJ2ErogKclQ3hggHrFsUzfMzEWXvOyC-9cvzh23HWiH6chQJDjdzC48vA8e-TI3CS-ooI8UxDpF3wTWTNLUA May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:48:11.599967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/af776ddc4c0f4e6390301a02c6bbfca0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: af776ddc4c0f4e6390301a02c6bbfca0 May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1628106124 May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: baac9b5e923747e6986720e3ac849056 May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1628106124-project May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance b3484367-cf43-4128-b7f2-96124543234c could not be found. May 24 10:48:11.601724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'b3484367-cf43-4128-b7f2-96124543234c'} May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b3484367-cf43-4128-b7f2-96124543234c could not be found. May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:48:11.602731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b3484367-cf43-4128-b7f2-96124543234c could not be found. May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b3484367-cf43-4128-b7f2-96124543234c could not be found. May 24 10:48:11.604246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:11.605614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] Returning 404 to user: Instance b3484367-cf43-4128-b7f2-96124543234c could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:48:11.605614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f6b8ffe0-d899-4b15-aa02-8ac3d2b88025 tempest-ServerRescueNegativeTestJSON-1628106124 tempest-ServerRescueNegativeTestJSON-1628106124-project] 10.222.0.35 "GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c" status: 404 len: 111 microversion: 2.1 time: 0.029349 May 24 10:48:11.605614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1178/2049] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:11 2022] GET /compute/v2.1/servers/b3484367-cf43-4128-b7f2-96124543234c => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:48:12.589287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:12.590915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:12.600445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:12.600922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:12.601930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:12.602164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:12.821014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:12.828475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f44fd40c-082b-46d0-ac1a-ff988f8d939b tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.243886 May 24 10:48:12.829043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1179/2050] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:12 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:13.231292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-6e699958-1f98-416b-a371-13b246eb577e req-2e837884-db3c-4d44-a64e-d4459318b8d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4cd27eab-afa0-46ce-810a-1985984a6034", "name": "network-vif-plugged", "status": "completed", "tag": "d81c2027-2c87-4fa4-9867-00e0f0f82878"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:13.240097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-6e699958-1f98-416b-a371-13b246eb577e req-2e837884-db3c-4d44-a64e-d4459318b8d4 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:13.242253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-6e699958-1f98-416b-a371-13b246eb577e req-2e837884-db3c-4d44-a64e-d4459318b8d4 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:13.252312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-6e699958-1f98-416b-a371-13b246eb577e req-2e837884-db3c-4d44-a64e-d4459318b8d4 service nova] Creating event network-vif-plugged:d81c2027-2c87-4fa4-9867-00e0f0f82878 for instance 4cd27eab-afa0-46ce-810a-1985984a6034 on hv-cinder-84311 May 24 10:48:13.258290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-6e699958-1f98-416b-a371-13b246eb577e req-2e837884-db3c-4d44-a64e-d4459318b8d4 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089845 May 24 10:48:13.258898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1180/2051] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:48:13.849876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:13.850731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:13.857415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:13.857866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:13.858656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:13.859089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:14.068042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:14.075742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1365285d-f7db-4e5a-b4fb-325ca9c138df tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.232972 May 24 10:48:14.076536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1181/2052] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:13 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:15.094228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:15.094891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:15.096905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:15.098172 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=8 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:15.103833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:15.104289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:15.107489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:15.107966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:15.349694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:15.357179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3a144237-be72-4640-908f-3893c9e281d4 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1319 microversion: 2.1 time: 0.266899 May 24 10:48:15.357799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1182/2053] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:15 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1319 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:16.374236 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:16.374869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:16.381622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:16.382066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:16.382846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:16.383276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:16.521872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:16.529351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3c49ed4b-d1ff-4d2d-8961-1199d60680eb tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1509 microversion: 2.1 time: 0.158956 May 24 10:48:16.529932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1183/2054] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:16 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1509 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:48:16.546571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9364204a-3b0b-42c8-95b4-4498101aaf06", "device": "/dev/sdb"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:16.547681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:16.554598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:16.555038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:16.555797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:16.556228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:17.299364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-95fd5469-e2a7-422d-a3a2-bd6e0408d606 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "POST /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.756694 May 24 10:48:17.299973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1184/2055] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:48:16 2022] POST /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034/os-volume_attachments => generated 194 bytes in 758 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:48:23.120384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d8e56212-733e-4a09-9bee-e74580116206 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'create', calling method: .version_select at 0x7f8732329430>, body: {"keypair": {"name": "tempest-keypair-789264052", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHP0MPaOUeKpVIGJJ0iIpYGN0cxKpKeEpBBxOpUn6gceEYHjk1G1LIgQ7CkYKMWnvmxt3CWGJHcUQSNFYm/3mn7Rt4ERi/2W45KU7C41cRe2pkiaCaN5tRWtrxGTAh8zVw=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:23.135206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-d8e56212-733e-4a09-9bee-e74580116206 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for project 107c9dab6bd24705b50f8d30181d6582. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:48:23.139274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-d8e56212-733e-4a09-9bee-e74580116206 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for user 0e46c23c8e3946238795b800b921380c and project 107c9dab6bd24705b50f8d30181d6582. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:48:23.159604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=4,nova_api:UPDATE=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:23.251495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d8e56212-733e-4a09-9bee-e74580116206 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.268704 May 24 10:48:23.252406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1185/2056] 10.222.0.35 () {62 vars in 1199 bytes} [Tue May 24 10:48:22 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 270 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) May 24 10:48:26.384888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-96323340", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c2b5ed2b-2e55-4176-a792-ce09df15f3ca"}], "security_groups": [{"name": "tempest-securitygroup--294392276"}], "key_name": "tempest-keypair-789264052", "user_data": "}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:26.565930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] validate_networks() for [('c2b5ed2b-2e55-4176-a792-ce09df15f3ca', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:48:27.212305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=55,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:27.224483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:48:27.224895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:48:27.225411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:48:27.225821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:48:27.226229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:48:27.235094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:27.235537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:27.236331 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:27.236764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:27.253624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for project 107c9dab6bd24705b50f8d30181d6582. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:48:27.258380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for user 0e46c23c8e3946238795b800b921380c and project 107c9dab6bd24705b50f8d30181d6582. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:48:27.269172 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:48:27.271669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:48:27.445793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4e07e300-5707-470b-a200-33c68bd92b68 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.068330 May 24 10:48:27.447252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1186/2057] 10.222.0.35 () {62 vars in 1187 bytes} [Tue May 24 10:48:26 2022] POST /compute/v2.1/servers => generated 401 bytes in 1070 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) May 24 10:48:27.457896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c91ffefd-58a8-4ac5-a009-6bd83dbf077d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:27.457896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c91ffefd-58a8-4ac5-a009-6bd83dbf077d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:27.514205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c91ffefd-58a8-4ac5-a009-6bd83dbf077d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:27.523637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c91ffefd-58a8-4ac5-a009-6bd83dbf077d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:27.526709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c91ffefd-58a8-4ac5-a009-6bd83dbf077d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:27.527828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c91ffefd-58a8-4ac5-a009-6bd83dbf077d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1279 microversion: 2.20 time: 0.075511 May 24 10:48:27.528381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1187/2058] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:27 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1279 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:28.549178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:28.549969 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:28.558000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:28.562042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:28.562042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:28.562042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:28.624508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:28.633609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:28.635148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:28.636299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ed7738fc-c83f-43d2-ae99-f5a95a17f519 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1283 microversion: 2.20 time: 0.092373 May 24 10:48:28.636868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1188/2059] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:28 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1283 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:29.656002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:29.656722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:29.666398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:29.667129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:29.668353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:29.668981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:29.733094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:29.741542 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:29.743081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:29.747704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0cbd3185-fc0f-41c8-aa7d-ed1e4eee1d8b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1283 microversion: 2.20 time: 0.093336 May 24 10:48:29.747704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1189/2060] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:29 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1283 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:48:30.762885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:30.762885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:30.769595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:30.771195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:30.771998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:30.772456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:30.846210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:30.857124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:30.858710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:30.859812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a30d0e42-14c8-48d9-9c49-2e33f2d08e94 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1349 microversion: 2.20 time: 0.101846 May 24 10:48:30.860394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1190/2061] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:30 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1349 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:31.877010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:31.877690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:31.884545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:31.884997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:31.885797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:31.886229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:32.054199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:32.063046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:32.064702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:32.066374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5d9350df-df87-400b-8dde-21953dea7c88 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.193268 May 24 10:48:32.067474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1191/2062] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:31 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:32.399353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-e6d23875-94b7-4949-a053-c0cfb8272a88 req-4ea6b65d-8c6e-476d-a544-fc26215a3095 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:32.408289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-e6d23875-94b7-4949-a053-c0cfb8272a88 req-4ea6b65d-8c6e-476d-a544-fc26215a3095 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:32.408721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-e6d23875-94b7-4949-a053-c0cfb8272a88 req-4ea6b65d-8c6e-476d-a544-fc26215a3095 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:32.420437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-e6d23875-94b7-4949-a053-c0cfb8272a88 req-4ea6b65d-8c6e-476d-a544-fc26215a3095 service nova] Creating event network-changed:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:48:32.425628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-e6d23875-94b7-4949-a053-c0cfb8272a88 req-4ea6b65d-8c6e-476d-a544-fc26215a3095 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.160455 May 24 10:48:32.427725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1192/2063] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:48:33.086019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:33.086709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:33.093776 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:33.094224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:33.094974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:33.095403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:33.228876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:33.236657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:33.238173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:33.239886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2fa2649c-2ca9-4c51-aaaf-512bf792786d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.158754 May 24 10:48:33.240442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1193/2064] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:33 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:34.257582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:34.258204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:34.264721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:34.265168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:34.265960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:34.266392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:34.401043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:34.408980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:34.411017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:34.416399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-83b27334-3275-440a-a108-0b4d655be6ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.158768 May 24 10:48:34.416399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1194/2065] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:34 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:35.430624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:35.431280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:35.437359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:35.437812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:35.438571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:35.439006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:35.569511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:35.577512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:35.579077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:35.580185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-45dff3a0-35a1-4fde-9650-55f98d23eec4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.154126 May 24 10:48:35.580740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1195/2066] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:35 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:36.597438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:36.598140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:36.605116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:36.605657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:36.606432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:36.606871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:36.742517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:36.751362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:36.753135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:36.754971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c4bb225c-0727-44a8-b846-e4d2cda63c8e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.160979 May 24 10:48:36.755928 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1196/2067] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:36 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:37.773362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:37.774051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:37.776850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:37.777575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=12 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:37.781670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:37.782120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:37.782878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:37.783307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:37.924190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:37.932897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:37.934649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:37.935847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f211af08-6181-4ba7-9645-a28e1b1ebb21 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.167097 May 24 10:48:37.937310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1197/2068] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:37 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:38.642946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:38.643438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:38.650149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:38.650597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:38.651354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:38.651779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:38.735255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lazy-loading 'services' on Instance uuid 4cd27eab-afa0-46ce-810a-1985984a6034 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:48:38.933976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e79389bc-f107-4045-99b9-ecb9738b7b3a tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034/os-volume_attachments/9364204a-3b0b-42c8-95b4-4498101aaf06" status: 202 len: 0 microversion: 2.1 time: 0.295162 May 24 10:48:38.934919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1198/2069] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:48:38 2022] DELETE /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034/os-volume_attachments/9364204a-3b0b-42c8-95b4-4498101aaf06 => generated 0 bytes in 297 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:48:38.959246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:38.965321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:38.966569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:38.967015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:38.967786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:38.968198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:39.102563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:39.110624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:39.112120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:39.113225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8bdc08a6-da9c-452d-ba6a-ea6f6d983964 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.158518 May 24 10:48:39.114585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1199/2070] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:38 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:39.653943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-5601ed69-b326-40df-9d97-744adf6b0ab0 req-7d8bdd6f-b8b7-4214-b162-9de321771e7b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "name": "network-vif-plugged", "status": "completed", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:39.663094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5601ed69-b326-40df-9d97-744adf6b0ab0 req-7d8bdd6f-b8b7-4214-b162-9de321771e7b service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:39.663603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5601ed69-b326-40df-9d97-744adf6b0ab0 req-7d8bdd6f-b8b7-4214-b162-9de321771e7b service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:39.674920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-5601ed69-b326-40df-9d97-744adf6b0ab0 req-7d8bdd6f-b8b7-4214-b162-9de321771e7b service nova] Creating event network-vif-plugged:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:48:39.680841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-5601ed69-b326-40df-9d97-744adf6b0ab0 req-7d8bdd6f-b8b7-4214-b162-9de321771e7b service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030700 May 24 10:48:39.681466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1200/2071] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:48:40.132678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:40.133394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:40.139567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:40.140256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:40.141029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:40.141485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:40.279459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:40.287613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:40.289108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:40.290253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6eca598c-f4bd-4ff8-bcc0-69dbd30db3ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.163024 May 24 10:48:40.290817 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1201/2072] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:40 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:41.110226 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:41.110724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:41.116960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:41.119586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:41.119586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:41.119586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:41.143840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:48:41.454710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bfdb482d-80f1-4076-8f82-133e2d59d8e3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "DELETE /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 204 len: 0 microversion: 2.1 time: 0.347992 May 24 10:48:41.455384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1202/2073] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:48:41 2022] DELETE /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:48:41.461545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:41.466392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:41.468478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:41.468957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:41.469749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:41.470182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:41.475109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:41.607978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:41.616048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:41.617631 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:41.618733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2f8e8f27-58d6-4788-acb7-518a279b2fd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1404 microversion: 2.20 time: 0.162209 May 24 10:48:41.619296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1203/2074] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:41 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1404 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:48:41.623584 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:41.624187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:41.633383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:41.633840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:41.634681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:41.635150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:41.772683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:41.780165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-007a8864-71d5-4847-9bde-26279a93bdc3 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1515 microversion: 2.1 time: 0.159933 May 24 10:48:41.780730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1204/2075] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:41 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1515 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:42.638257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:42.638898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:42.647705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:42.648156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:42.648923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:42.649379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:42.789953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:42.797942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:42.799423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:42.800593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b3966f5b-febc-41cc-8edf-ce9aac4b6cf6 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1597 microversion: 2.20 time: 0.166279 May 24 10:48:42.801199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1205/2076] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:48:42 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1597 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:48:42.805597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:42.806304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:42.811980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:42.812475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:42.822752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:42.822752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:42.953919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:42.963227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-80433425-60e4-4507-b595-d491c9329982 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1515 microversion: 2.1 time: 0.160850 May 24 10:48:42.963832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1206/2077] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:42 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1515 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:42.968703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:42.969408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:42.977332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:42.977726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:42.978482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:42.978916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:43.063074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7104c71d-af8f-4504-b09d-c85478e2f5e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.098150 May 24 10:48:43.063772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1207/2078] 10.222.0.35 () {60 vars in 1315 bytes} [Tue May 24 10:48:42 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) May 24 10:48:43.949599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-c1449d4f-753a-4601-9e93-95b0b4fed75a req-e3da8a67-c860-49b9-b798-3ab1ca35f775 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4cd27eab-afa0-46ce-810a-1985984a6034", "name": "network-vif-deleted", "tag": "d81c2027-2c87-4fa4-9867-00e0f0f82878"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:43.960461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-c1449d4f-753a-4601-9e93-95b0b4fed75a req-e3da8a67-c860-49b9-b798-3ab1ca35f775 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:43.961095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-c1449d4f-753a-4601-9e93-95b0b4fed75a req-e3da8a67-c860-49b9-b798-3ab1ca35f775 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:43.973740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-c1449d4f-753a-4601-9e93-95b0b4fed75a req-e3da8a67-c860-49b9-b798-3ab1ca35f775 service nova] Creating event network-vif-deleted:d81c2027-2c87-4fa4-9867-00e0f0f82878 for instance 4cd27eab-afa0-46ce-810a-1985984a6034 on hv-cinder-84311 May 24 10:48:43.979095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-c1449d4f-753a-4601-9e93-95b0b4fed75a req-e3da8a67-c860-49b9-b798-3ab1ca35f775 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033712 May 24 10:48:43.979683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1208/2079] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:48:43.988123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:43.989175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:43.996442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:43.996897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:43.997707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:43.998140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:44.098806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb2d87f281834e4697a8c3559b6e83a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f0441599dc5456da3e0f48be7dd2d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:48:44.109123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1d70f8cb-c50f-4736-bb23-6aefed6cd592 tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 200 len: 1473 microversion: 2.1 time: 0.124674 May 24 10:48:44.109920 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1209/2080] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:43 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 1473 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:48:44.115349 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-c589c993-d66d-4bbc-b355-1bfec1153a77 req-f4a32c1a-671e-472a-a1af-69e33692a9dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:48:44.124295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-c589c993-d66d-4bbc-b355-1bfec1153a77 req-f4a32c1a-671e-472a-a1af-69e33692a9dc service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:44.124818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-c589c993-d66d-4bbc-b355-1bfec1153a77 req-f4a32c1a-671e-472a-a1af-69e33692a9dc service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:44.136337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-c589c993-d66d-4bbc-b355-1bfec1153a77 req-f4a32c1a-671e-472a-a1af-69e33692a9dc service nova] Creating event network-changed:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:48:44.141857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-c589c993-d66d-4bbc-b355-1bfec1153a77 req-f4a32c1a-671e-472a-a1af-69e33692a9dc service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030199 May 24 10:48:44.142568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1210/2081] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:48:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:48:45.130922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:48:45.131620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] [instance: 4cd27eab-afa0-46ce-810a-1985984a6034] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:48:45.139399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:45.152403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:45.152403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:48:45.152403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:48:45.166975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] HTTP exception thrown: Instance 4cd27eab-afa0-46ce-810a-1985984a6034 could not be found. May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Request method failure captured: May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 HTTP/1.1 May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLBVl3Urd-y_lJGU7mpRi910LZ9wevShzCRu_NwiSUYMpfxObJwflmsI4Of-vghkXEmbN7k4JVoMQJzBeU_ObpcHO1pW9yFA2YGSnXzr1eiYFxuZfPbxkO_2ZUypBbR3eooiJ-8kLYfYX1w55TYzcxHrfsTiB38kcF6Xl4psnIPY_Kw May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:48:45.168562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/0f0441599dc5456da3e0f48be7dd2d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 0f0441599dc5456da3e0f48be7dd2d20 May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1458325324 May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: bb2d87f281834e4697a8c3559b6e83a2 May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-VolumesSnapshotTestJSON-1458325324-project May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 4cd27eab-afa0-46ce-810a-1985984a6034 could not be found. May 24 10:48:45.170135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '4cd27eab-afa0-46ce-810a-1985984a6034'} May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4cd27eab-afa0-46ce-810a-1985984a6034 could not be found. May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:48:45.171101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4cd27eab-afa0-46ce-810a-1985984a6034 could not be found. May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4cd27eab-afa0-46ce-810a-1985984a6034 could not be found. May 24 10:48:45.172530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:48:45.175587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] Returning 404 to user: Instance 4cd27eab-afa0-46ce-810a-1985984a6034 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:48:45.175587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a6affc49-deea-4773-81a2-1ff47d0f6bfd tempest-VolumesSnapshotTestJSON-1458325324 tempest-VolumesSnapshotTestJSON-1458325324-project] 10.222.0.35 "GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034" status: 404 len: 111 microversion: 2.1 time: 0.044600 May 24 10:48:45.179810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1211/2082] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:48:45 2022] GET /compute/v2.1/servers/4cd27eab-afa0-46ce-810a-1985984a6034 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:49:20.452953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:20.453692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:20.455830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:20.457064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=38 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:20.462511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:20.463672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:20.464467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:20.464896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:20.488394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4067ed4f-7a35-4bb7-86d7-5dd23ef915ce tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/ips" status: 200 len: 153 microversion: 2.20 time: 0.039526 May 24 10:49:20.489009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1212/2083] 10.222.0.35 () {60 vars in 1288 bytes} [Tue May 24 10:49:20 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/ips => generated 153 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:49:23.092427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:49:23.093079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:23.100184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:23.100708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:23.101512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:23.101945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:23.125513 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lazy-loading 'flavor' on Instance uuid ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:49:23.664497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e3f36a17-1816-46ef-b492-83574b717ddc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/action" status: 202 len: 0 microversion: 2.20 time: 0.576669 May 24 10:49:23.665191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1213/2084] 10.222.0.35 () {62 vars in 1318 bytes} [Tue May 24 10:49:23 2022] POST /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/action => generated 0 bytes in 578 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 24 10:49:23.675105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:23.675804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:23.683348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:23.683795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:23.684571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:23.684999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:23.818128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:23.826261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:23.827748 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:23.828897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7d27f7a7-d3cc-4590-8dbd-7b7d86d1770a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.158281 May 24 10:49:23.829482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1214/2085] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:23 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:24.848082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:24.848783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:24.855740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:24.856185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:24.856941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:24.857399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:24.987203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:24.995628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:24.997119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:24.998286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-60a2cd9d-10a0-47f8-8a8b-5a557bddd886 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.154265 May 24 10:49:24.998847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1215/2086] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:24 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:26.016362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:26.017086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:26.023482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:26.023932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:26.024695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:26.025661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:26.151681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:26.159551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:26.161052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:26.162231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8eb23079-6c9c-4aca-afc5-2ad28b66f502 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.149809 May 24 10:49:26.162793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1216/2087] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:26 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:27.182583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:27.185527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:27.189869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:27.190317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:27.191075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:27.191509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:27.317833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:27.326294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:27.327818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:27.328973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-657b8237-b0fb-4896-b01f-2755efb0e60e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.151126 May 24 10:49:27.329572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1217/2088] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:27 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:28.347763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:28.348445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:28.355654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:28.356109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:28.356869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:28.357332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:28.489908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:28.498022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:28.499519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:28.500674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-58ad145f-eff9-4d99-b052-c6a7b083f9b8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.157012 May 24 10:49:28.501233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1218/2089] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:28 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:29.518555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:29.519175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:29.525834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:29.528672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:29.528672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:29.528672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:29.655319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:29.663348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:29.665918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:29.666696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-94c5f21d-d4e8-4590-a459-611d41653049 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.152079 May 24 10:49:29.667256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1219/2090] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:29 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:30.685399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:30.686073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:30.692460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:30.692905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:30.693707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:30.694135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:30.825704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:30.833557 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:30.835044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:30.836444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-579c2d63-958d-4bca-a168-e38e83cbc2de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.155303 May 24 10:49:30.837000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1220/2091] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:30 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:31.855200 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:31.855862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:31.862092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:31.862534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:31.863279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:31.863742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:31.990273 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:31.998765 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:32.000279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:32.001473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-165da387-69c4-4778-a1e8-9dc1ead08d61 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.150547 May 24 10:49:32.002035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1221/2092] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:31 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:33.021094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:33.021801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:33.028712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:33.029160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:33.029947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:33.030382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:33.169627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:33.177357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:33.178844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:33.179984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e0a74cd4-ccae-4a01-9295-42fdd35fddd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.162831 May 24 10:49:33.180747 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1222/2093] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:33 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:34.199360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:34.199982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:34.206527 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:34.206980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:34.207746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:34.208182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:34.338420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:34.346358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:34.347857 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:34.349010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bb63a92a-ba73-45bf-964b-41e51673ebae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.154459 May 24 10:49:34.349779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1223/2094] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:34 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:35.367520 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:35.368229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:35.374336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:35.374787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:35.375545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:35.375980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:35.500378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:35.508314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:35.509850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:35.511559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1f69dcb0-6c6a-4e65-9223-0f058f7bf8dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1743 microversion: 2.20 time: 0.147985 May 24 10:49:35.512121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1224/2095] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:35 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1743 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:36.529714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:36.530351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:36.536797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:36.537252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:36.538043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:36.538478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:36.673093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:36.681295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:36.683751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:36.684995 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-39945e4f-08e1-433d-bcb7-d49dabb29920 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.159172 May 24 10:49:36.685621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1225/2096] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:36 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:37.704437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:37.705117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:37.711939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:37.712643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:37.713621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:37.714211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:37.877669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:37.885473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:37.894409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:37.896211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:37.897544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-332d8eaf-184a-4eea-a45a-7dc59291aaa3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.197915 May 24 10:49:37.898351 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1226/2097] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:37 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:37.908268 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-7b011c0f-1d76-486d-8011-35d78c2225b9 req-d3292af1-6b0e-4a98-8661-5941531041f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "name": "network-vif-plugged", "status": "completed", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:49:37.917692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7b011c0f-1d76-486d-8011-35d78c2225b9 req-d3292af1-6b0e-4a98-8661-5941531041f5 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:37.918221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7b011c0f-1d76-486d-8011-35d78c2225b9 req-d3292af1-6b0e-4a98-8661-5941531041f5 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:37.929855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-7b011c0f-1d76-486d-8011-35d78c2225b9 req-d3292af1-6b0e-4a98-8661-5941531041f5 service nova] Creating event network-vif-plugged:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:49:37.936087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-7b011c0f-1d76-486d-8011-35d78c2225b9 req-d3292af1-6b0e-4a98-8661-5941531041f5 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036100 May 24 10:49:37.936692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1227/2098] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:49:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:49:38.917415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:38.919887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:38.928107 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:38.928601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:38.929414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:38.929993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:39.078214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:39.087265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:39.088918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:39.090238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a9db34b-aa34-49c0-a4fe-413262a5b719 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.177480 May 24 10:49:39.091267 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1228/2099] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:38 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:40.110074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:40.110950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:40.117535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:40.122835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:40.122835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:40.122835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:40.253189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:40.261408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:40.262979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:40.264133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1610ddc4-6aa7-43b9-b8e4-2b115e9824cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.159014 May 24 10:49:40.264699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1229/2100] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:40 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:49:41.282837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:41.283494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:41.290558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:41.291417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:41.292456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:41.292960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:41.442142 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:41.451261 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:41.452849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:41.454160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4c022954-e463-4d39-9456-4875dbc42349 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.175272 May 24 10:49:41.464978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1230/2101] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:41 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:42.472390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:42.473104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:42.480425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:42.480990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:42.482731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:42.483218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:42.629977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:42.639659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:42.641660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:42.643288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7d68f1fc-bec0-48aa-ad75-243504e29c77 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.174554 May 24 10:49:42.644071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1231/2102] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:42 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:49:43.662462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:43.663126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:43.670245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:43.670782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:43.671532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:43.671963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:43.824106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:43.832821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:43.834593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:43.836477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aefb12ec-1677-438e-b1c2-5a3e1a88ae96 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.178007 May 24 10:49:43.837134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1232/2103] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:43 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:44.854660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:44.856041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:44.863297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:44.863816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:44.864587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:44.865019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:45.010338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:45.019840 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:45.021720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:45.023106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b803efbb-6374-44e7-94ab-a0b5d8f2b7f3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.172400 May 24 10:49:45.023899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1233/2104] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:44 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:46.041868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:46.043286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:46.050099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:46.050577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:46.051324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:46.051755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:46.198859 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:46.209006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:46.210831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:46.212063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-092465e9-3351-48d9-900c-1361ca785ba1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.174227 May 24 10:49:46.212651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1234/2105] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:46 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:47.230674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:47.232789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:47.242933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:47.243659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:47.244623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:47.245054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:47.384033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:47.394316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:47.395816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:47.396973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-62ada8e0-210b-4b72-97cb-58824137ca4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.170293 May 24 10:49:47.397571 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1235/2106] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:47 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:48.416262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:48.416974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:48.423974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:48.424416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:48.425166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:48.425622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:48.558102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:48.565944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:48.567429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:48.568573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e670a628-4db1-4527-aeb0-d6e4621e532b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1738 microversion: 2.20 time: 0.156235 May 24 10:49:48.569124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1236/2107] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:48 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1738 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:49.587823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:49.588515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:49.595475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:49.595923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:49.596673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:49.597101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:49.722728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:49.730690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:49.732184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:49.733359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bf3f8db6-3646-4075-b748-49af0c4b0656 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1720 microversion: 2.20 time: 0.149671 May 24 10:49:49.733926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1237/2108] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:49 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:50.751556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:50.752258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:50.758774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:50.759221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:50.759985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:50.760418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:51.109997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:51.118154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:51.119781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:51.120948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6690a3d7-55e0-44bd-a478-044cf341a4a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1720 microversion: 2.20 time: 0.373442 May 24 10:49:51.121555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1238/2109] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:50 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1720 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:51.876952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-331bea71-acd3-46dc-885c-44513a05be92 req-e9ba0e7a-0066-4496-ab48-443437e15fe8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "name": "network-vif-unplugged", "status": "completed", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:49:51.885763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-331bea71-acd3-46dc-885c-44513a05be92 req-e9ba0e7a-0066-4496-ab48-443437e15fe8 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:51.886202 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-331bea71-acd3-46dc-885c-44513a05be92 req-e9ba0e7a-0066-4496-ab48-443437e15fe8 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:51.897752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-331bea71-acd3-46dc-885c-44513a05be92 req-e9ba0e7a-0066-4496-ab48-443437e15fe8 service nova] Creating event network-vif-unplugged:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:49:51.902888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-331bea71-acd3-46dc-885c-44513a05be92 req-e9ba0e7a-0066-4496-ab48-443437e15fe8 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029683 May 24 10:49:51.903482 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1239/2110] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:49:51 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:49:52.141179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:52.141793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:52.148138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:52.152152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:52.152152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:52.152152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:52.277249 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:52.285152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:52.286678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:52.287826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f1e3ab0b-12b4-43dd-a9eb-50b9fc6a154c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1663 microversion: 2.20 time: 0.151828 May 24 10:49:52.288395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1240/2111] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:52 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1663 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:52.305059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "464d2540-aa92-4092-96e4-f9e224ce16ae"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:49:52.306198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:52.312625 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:52.313074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:52.313864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:52.314303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:53.136052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3d620262-117e-45fc-a68a-9158c31b0cb4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.834612 May 24 10:49:53.136711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1241/2112] 10.222.0.35 () {62 vars in 1363 bytes} [Tue May 24 10:49:52 2022] POST /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments => generated 188 bytes in 836 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:49:53.250154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:49:53.250713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:53.253339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=61 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:53.258340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:53.258877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:53.259622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:53.260050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:53.264821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:53.516413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-246eacac-b462-48b2-aba6-0a8801e8f4e8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/action" status: 202 len: 0 microversion: 2.20 time: 0.270141 May 24 10:49:53.517093 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1242/2113] 10.222.0.35 () {62 vars in 1318 bytes} [Tue May 24 10:49:53 2022] POST /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/action => generated 0 bytes in 272 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 24 10:49:53.524757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:53.525383 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:53.532291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:53.532730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:53.533521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:53.533952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:53.668143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:53.676850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:53.678360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:53.679608 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7f76583b-92b7-492e-9728-afc78cea0dfe tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1749 microversion: 2.20 time: 0.158567 May 24 10:49:53.680160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1243/2114] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:53 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1749 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:54.698130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:54.698766 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:54.705792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:54.706282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:54.707036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:54.707461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:54.834957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:54.844835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:54.846384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:54.847541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f0eda553-c7b4-4661-ae5b-0f6ce781271a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1749 microversion: 2.20 time: 0.153474 May 24 10:49:54.848151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1244/2115] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:54 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1749 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:55.866164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:55.866841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:55.873781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:55.874420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:55.875208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:55.875641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:56.014847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:56.024938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:56.030147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:56.030147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-08043430-f8a4-4fe7-9076-cd7d55a8e706 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1751 microversion: 2.20 time: 0.165894 May 24 10:49:56.030147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1245/2116] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:55 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1751 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:57.050164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:57.050791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:57.057237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:57.057725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:57.058492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:57.058929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:57.196863 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:57.206238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:57.207739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:57.208903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-56c75be5-9a73-48c0-8b24-81b17d9db261 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.163625 May 24 10:49:57.209490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1246/2117] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:57 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:49:58.227947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:58.229311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:58.246412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:58.248060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:58.250329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:58.252272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:58.454519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:58.465848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:58.467633 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:58.470109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0ba623e3-6da5-4448-9d8a-e0551e0eae26 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.245149 May 24 10:49:58.483796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1247/2118] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:58 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:49:59.491126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:49:59.492602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:49:59.499904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:59.500477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:59.501742 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:49:59.502305 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:49:59.655173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:59.665209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:59.666958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:49:59.668232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ea149fd1-4ebf-4b6f-8d76-90f09f8a52dc tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.181140 May 24 10:49:59.668795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1248/2119] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:49:59 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:00.686966 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:00.687653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:00.695078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:00.695524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:00.696292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:00.696730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:00.855500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:00.865581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:00.867559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:00.868808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a9f4d6c6-dcf8-4c4a-8c8d-378536a43822 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.185739 May 24 10:50:00.880104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1249/2120] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:00 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:01.887065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:01.887693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:01.894153 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:01.894596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:01.895344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:01.895771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:02.024198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:02.032475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:02.034783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:02.035933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0d10e826-4957-4faa-8105-0aec23478f4a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.152829 May 24 10:50:02.036495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1250/2121] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:01 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:03.054492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:03.055180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:03.061660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:03.062112 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:03.062878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:03.063314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:03.213629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:03.222779 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:03.224282 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:03.225477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4428c9df-a2c9-424e-9098-23dd2c821ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.174850 May 24 10:50:03.226038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1251/2122] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:03 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:04.244152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:04.244843 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:04.251332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:04.252620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:04.253557 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:04.254000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:04.388694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:04.406543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:04.408405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:04.409667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e35a8023-8d74-462a-8ce4-bd4669000371 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.169332 May 24 10:50:04.410823 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1252/2123] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:04 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:05.431185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:05.432660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:05.440354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:05.441234 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:05.442283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:05.442941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:05.592311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:05.601945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:05.603820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:05.605000 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cb35bfca-7141-484c-9320-70b363fec374 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.178490 May 24 10:50:05.605876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1253/2124] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:05 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:06.623356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:06.624082 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:06.630646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:06.631119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:06.632029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:06.632461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:06.762216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:06.780744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:06.783719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:06.785066 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b66a88a6-fa45-499a-b371-f7528603b299 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.165705 May 24 10:50:06.786204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1254/2125] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:06 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:07.806337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:07.807122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:07.818201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:07.826828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:07.826828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:07.826828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:07.968865 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:07.977444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:07.979049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:07.980211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6b33a16f-29d3-4e8f-bbdf-10e0012f62ec tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.177921 May 24 10:50:07.980758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1255/2126] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:07 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:50:08.999488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:09.000141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:09.006868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:09.007364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:09.008121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:09.008551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:09.135104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:09.143705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:09.145294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:09.146449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6eae49dd-8fd9-4f27-9f6b-47f2f0d47bca tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.150900 May 24 10:50:09.147007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1256/2127] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:08 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:10.164948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:10.165590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:10.172251 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:10.172690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:10.173814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:10.174243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:10.304933 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:10.308350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:10.313701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:10.315478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:10.316678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f2e2a880-ba9e-4482-afcc-bea077f1ad8d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.155787 May 24 10:50:10.317475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1257/2128] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:10 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:11.336345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:11.337150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:11.343959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:11.344302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:11.345222 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:11.345869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:11.583088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:11.583088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:11.583432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:11.583432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ef6882fc-ccd3-4e6e-94f3-a2639a952cf0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.159863 May 24 10:50:11.583432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1258/2129] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:11 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:12.511946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:12.512590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:12.520338 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:12.520788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:12.521576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:12.522022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:12.652441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:12.661909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:12.663439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:12.664592 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7b181146-0e36-405f-a7bb-5b821485c20f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.156546 May 24 10:50:12.665162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1259/2130] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:12 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:13.683416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:13.684026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:13.691098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:13.691543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:13.692296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:13.692720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:13.820382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:13.828632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:13.830277 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:13.831450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3830233d-3b51-41bb-9dbe-758dee0de42b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.151971 May 24 10:50:13.832004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1260/2131] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:13 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:14.850105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:14.850837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:14.857110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:14.857579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:14.858329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:14.858758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:14.984364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:14.993521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:14.995022 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:14.996166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-67268969-d67e-4e07-8ea6-2767567375a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.149940 May 24 10:50:14.996724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1261/2132] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:14 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:16.014321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:16.014971 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:16.021478 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:16.021923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:16.022676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:16.023104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:16.152097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:16.161034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:16.162743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:16.163901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-205f8e54-31e1-42e3-a472-75eed0d384a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.153480 May 24 10:50:16.164706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1262/2133] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:16 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:17.185344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:17.185939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:17.192990 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:17.193462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:17.194323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:17.194743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:17.320864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:17.329095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:17.330626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:17.331761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8e0db17c-628a-4d21-bff7-e07f2ab16b39 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.150305 May 24 10:50:17.332313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1263/2134] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:17 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:18.350591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:18.351265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:18.357711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:18.358158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:18.358915 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:18.359345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:18.489758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:18.498198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:18.499697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:18.500884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ae876fbe-2295-4eda-910a-0c0a698b1bd2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.154485 May 24 10:50:18.501474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1264/2135] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:18 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:19.520550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:19.521303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:19.527795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:19.528238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:19.529002 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:19.529468 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:19.662704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:19.671007 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:19.672491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:19.673678 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c6388706-81ec-4201-b09c-93875b72ffa1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.157665 May 24 10:50:19.674232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1265/2136] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:19 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:20.037147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-7a64f1ef-b764-4600-9796-6474c599f7a0 req-52cf1ee6-8bb8-4170-9752-fceaf4606c35 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "name": "network-vif-plugged", "status": "completed", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:50:20.045015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7a64f1ef-b764-4600-9796-6474c599f7a0 req-52cf1ee6-8bb8-4170-9752-fceaf4606c35 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:20.045514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7a64f1ef-b764-4600-9796-6474c599f7a0 req-52cf1ee6-8bb8-4170-9752-fceaf4606c35 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:20.058258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-7a64f1ef-b764-4600-9796-6474c599f7a0 req-52cf1ee6-8bb8-4170-9752-fceaf4606c35 service nova] Creating event network-vif-plugged:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:50:20.063308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-7a64f1ef-b764-4600-9796-6474c599f7a0 req-52cf1ee6-8bb8-4170-9752-fceaf4606c35 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029791 May 24 10:50:20.063868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1266/2137] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:50:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:50:20.692929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:20.693684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:20.699938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:20.700392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:20.701165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:20.701627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:20.835248 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:20.843534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:20.845023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:20.846762 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cce5d32c-e1e6-4578-8c39-e53ed33dda72 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.158743 May 24 10:50:20.847324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1267/2138] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:20 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:21.872460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:21.872460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:21.872460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:21.872460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:21.873610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:21.873610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:22.005656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:22.014139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:22.015639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:22.016796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0c5facde-4df4-45f1-bf29-d7a68f2b46ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.156806 May 24 10:50:22.017525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1268/2139] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:21 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:23.036232 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:23.036927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:23.043583 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:23.044024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:23.044789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:23.045217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:23.180440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:23.189046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:23.191024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:23.192191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-72dd089a-9db7-4383-8369-e932a0bfeee8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1807 microversion: 2.20 time: 0.160047 May 24 10:50:23.192764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1269/2140] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:23 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1807 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:24.211733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:24.212372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:24.214790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=58 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:24.219560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:24.220011 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:24.220772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:24.221212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:24.348199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:24.356822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:24.358339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:50:24.359498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0dc76b97-70b0-4860-96e8-5223d51c22cd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1794 microversion: 2.20 time: 0.151952 May 24 10:50:24.360063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1270/2141] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:24 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1794 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:50:56.996329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:56.997030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:56.999101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:56.999731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=74 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:57.004477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:57.004927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:57.006993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:57.007771 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:57.037539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-98ebef96-19ed-4ba5-a8cc-4603046d8ccf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments/464d2540-aa92-4092-96e4-f9e224ce16ae" status: 200 len: 194 microversion: 2.20 time: 0.045295 May 24 10:50:57.038043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1271/2142] 10.222.0.35 () {60 vars in 1453 bytes} [Tue May 24 10:50:56 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments/464d2540-aa92-4092-96e4-f9e224ce16ae => generated 194 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:50:57.100313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:57.100797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:57.107103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:57.107545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:57.108291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:57.108719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:57.189030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lazy-loading 'services' on Instance uuid ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:50:57.388576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a90225ca-351c-4359-b852-4f2fb1b42128 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments/464d2540-aa92-4092-96e4-f9e224ce16ae" status: 202 len: 0 microversion: 2.20 time: 0.291821 May 24 10:50:57.389169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1272/2143] 10.222.0.35 () {60 vars in 1456 bytes} [Tue May 24 10:50:57 2022] DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments/464d2540-aa92-4092-96e4-f9e224ce16ae => generated 0 bytes in 293 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 24 10:50:57.397721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:57.398716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:57.405902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:57.406352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:57.407118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:57.407549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:57.438981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c9d1faed-6d72-4f1d-9ecb-5a0d115e86cf tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045490 May 24 10:50:57.439539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1273/2144] 10.222.0.35 () {60 vars in 1342 bytes} [Tue May 24 10:50:57 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:50:58.451304 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:58.452390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:58.459706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:58.460156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:58.461179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:58.461864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:58.494926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5d34be83-22c5-4c07-8065-2fd13255171c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047060 May 24 10:50:58.495465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1274/2145] 10.222.0.35 () {60 vars in 1342 bytes} [Tue May 24 10:50:58 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:50:59.506646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:59.511209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:59.514117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:59.514569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:59.515323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:59.515755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:59.544914 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c17fd29a-8fc5-436e-a911-dca7ddc7d201 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.042210 May 24 10:50:59.545510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1275/2146] 10.222.0.35 () {60 vars in 1342 bytes} [Tue May 24 10:50:59 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:50:59.602296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:59.602831 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:59.609586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:59.610029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:59.610783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:59.611386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:59.637353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:50:59.921023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-28c6a2ba-bd54-4c7e-aa13-a84d760c29a7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 204 len: 0 microversion: 2.20 time: 0.322242 May 24 10:50:59.921667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1276/2147] 10.222.0.35 () {60 vars in 1279 bytes} [Tue May 24 10:50:59 2022] DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 24 10:50:59.929819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:50:59.930501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:50:59.936359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:59.936837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:50:59.937610 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:50:59.938042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:00.069796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:00.078296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:00.079770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:00.080911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a63effde-6ca2-47bd-8134-fad669aa2930 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.155056 May 24 10:51:00.081507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1277/2148] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:50:59 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:01.099612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:01.100292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:01.107734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:01.108179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:01.108932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:01.109398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:01.237407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:01.245722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:01.247224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:01.248376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bfd9f614-d0fe-41c7-92ca-0a869af67bf5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.152709 May 24 10:51:01.248965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1278/2149] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:01 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:02.268017 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:02.268721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:02.275485 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:02.275942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:02.276718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:02.277162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:02.411103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:02.420028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:02.421555 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:02.422716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-01956e43-939d-4e0b-8daa-30835edb3420 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1722 microversion: 2.20 time: 0.158730 May 24 10:51:02.423497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1279/2150] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:02 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:02.707722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-6f551ebf-1758-494e-9671-090729b50d3e req-417069b5-068b-4eaa-b3f2-9538067c5025 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac4af8cb-8880-4a4e-84d9-b4170d85e4a4", "name": "network-vif-deleted", "tag": "61224ab7-6884-4f4e-80ac-754d760250f7"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:02.717618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-6f551ebf-1758-494e-9671-090729b50d3e req-417069b5-068b-4eaa-b3f2-9538067c5025 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:02.718059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-6f551ebf-1758-494e-9671-090729b50d3e req-417069b5-068b-4eaa-b3f2-9538067c5025 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:02.729944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-6f551ebf-1758-494e-9671-090729b50d3e req-417069b5-068b-4eaa-b3f2-9538067c5025 service nova] Creating event network-vif-deleted:61224ab7-6884-4f4e-80ac-754d760250f7 for instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 on hv-cinder-84311 May 24 10:51:02.735301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-6f551ebf-1758-494e-9671-090729b50d3e req-417069b5-068b-4eaa-b3f2-9538067c5025 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031285 May 24 10:51:02.735880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1280/2151] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:51:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:51:03.442599 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:03.443309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:03.450793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:03.451243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:03.452027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:03.452470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:03.489336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:03.489792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:03.490252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lazy-loading 'fault' on Instance uuid ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:51:03.536059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:03.545076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:03.546613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:03.547858 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5c4687d1-af9f-4f3a-b000-9d1584d7ae8a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 200 len: 1369 microversion: 2.20 time: 0.110177 May 24 10:51:03.548424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1281/2152] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:03 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 1369 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:04.566974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:04.567709 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:04.574512 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:04.574959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:04.575715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:04.576150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:04.590248 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] HTTP exception thrown: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Request method failure captured: May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 HTTP/1.1 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLf24EQxpzEOxG0lwZ9oMlMGcLBXaR2jD6i-8F8R384A4Vuo1B6Nl8m6_z52Qw8ILr6ht3fR1X-fKF-LungyjKRkkGr-e2OgIIMZ6PUJ5-MNYpr1N2muUGkraoEydMWJ-7bQWpApKakXOe7XslJU301k65t3bTcC_SHe4dF8jpN84GA May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Openstack-Nova-Api-Version: 2.20 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:51:04.591875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 0e46c23c8e3946238795b800b921380c May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:51:04.593793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'ac4af8cb-8880-4a4e-84d9-b4170d85e4a4'} May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:51:04.594847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:51:04.596556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:51:04.598396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Returning 404 to user: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:51:04.598396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eca998f4-08fd-4142-818e-42b647193ce5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 404 len: 111 microversion: 2.20 time: 0.030195 May 24 10:51:04.598396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1282/2153] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:04 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 24 10:51:04.601649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4e1f77ce-349d-42a8-840b-f550205aed89 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '.version_select at 0x7f8732112b80>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:04.653824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4e1f77ce-349d-42a8-840b-f550205aed89 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-789264052" status: 204 len: 0 microversion: 2.20 time: 0.055743 May 24 10:51:04.654384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1283/2154] 10.222.0.35 () {60 vars in 1258 bytes} [Tue May 24 10:51:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-789264052 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 24 10:51:05.370189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8203ca92-e78f-43fd-b90d-f50fc008d2af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'create', calling method: .version_select at 0x7f87374dd160>, body: {"keypair": {"name": "tempest-keypair-1336007704", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCkodUCsXitd9qAqvqsnJPgCNYfXonED57GJa1OaXDuTz22w6jcRQXSBY4xMP9qAWy5bb+kD7Ns9JTR77bIEDnvxlMrKZyyZPhhlsknDv4gN+oFE2dD/5jY2ndguTXkqMw=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:05.387918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-8203ca92-e78f-43fd-b90d-f50fc008d2af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for project 107c9dab6bd24705b50f8d30181d6582. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:51:05.391620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-8203ca92-e78f-43fd-b90d-f50fc008d2af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for user 0e46c23c8e3946238795b800b921380c and project 107c9dab6bd24705b50f8d30181d6582. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:51:05.498924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8203ca92-e78f-43fd-b90d-f50fc008d2af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.133394 May 24 10:51:05.499560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1284/2155] 10.222.0.35 () {62 vars in 1199 bytes} [Tue May 24 10:51:05 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 135 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) May 24 10:51:07.748768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1526714453", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c2b5ed2b-2e55-4176-a792-ce09df15f3ca"}], "security_groups": [{"name": "tempest-securitygroup--993521060"}], "key_name": "tempest-keypair-1336007704", "user_data": "}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:07.850864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] validate_networks() for [('c2b5ed2b-2e55-4176-a792-ce09df15f3ca', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:51:08.248420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:51:08.248420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:51:08.248420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:51:08.248420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:51:08.248420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:51:08.249149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:08.249149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:08.249149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:08.249149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:08.273470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for project 107c9dab6bd24705b50f8d30181d6582. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:51:08.278367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Getting quotas for user 0e46c23c8e3946238795b800b921380c and project 107c9dab6bd24705b50f8d30181d6582. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:51:08.299352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:51:08.301657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:51:08.438103 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5ad7e59b-b5f4-4bdd-b3d8-f38c212d076e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.694328 May 24 10:51:08.438764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1285/2156] 10.222.0.35 () {62 vars in 1187 bytes} [Tue May 24 10:51:07 2022] POST /compute/v2.1/servers => generated 401 bytes in 696 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) May 24 10:51:08.450683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e3caf46d-3b04-4cf8-bc99-bcfa2764a38f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:08.451291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e3caf46d-3b04-4cf8-bc99-bcfa2764a38f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:08.518507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e3caf46d-3b04-4cf8-bc99-bcfa2764a38f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:08.527113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e3caf46d-3b04-4cf8-bc99-bcfa2764a38f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:08.529736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e3caf46d-3b04-4cf8-bc99-bcfa2764a38f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:08.530884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e3caf46d-3b04-4cf8-bc99-bcfa2764a38f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1282 microversion: 2.20 time: 0.084688 May 24 10:51:08.531621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1286/2157] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:08 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1282 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:51:09.549519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:09.550228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:09.560031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:09.560031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:09.560031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:09.560031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:09.622287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:09.630525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:09.632059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:09.633164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a9a06334-1cfd-45da-83f5-dfcbf0e08f18 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1286 microversion: 2.20 time: 0.087671 May 24 10:51:09.633830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1287/2158] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:09 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1286 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:10.652387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:10.653044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:10.659801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:10.660254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:10.661195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:10.661660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:10.724689 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:10.733793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:10.735563 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:10.736705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7c4d9bf3-adac-4c4d-8b34-fb37073f0e11 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1342 microversion: 2.20 time: 0.089075 May 24 10:51:10.737254 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1288/2159] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:10 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1342 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:51:11.754946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:11.755541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:11.762690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:11.763141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:11.763905 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:11.764345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:11.908738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:11.915423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:11.919774 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:11.921462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:11.922732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4cb4153-855a-41be-80e4-debf828605f4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1407 microversion: 2.20 time: 0.171604 May 24 10:51:11.923562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1289/2160] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:11 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1407 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:12.152052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-487e2921-a728-4e74-a2e5-90e6299752c7 req-5d9ff57e-d9fb-4d79-90db-b4d0f098c8ab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:12.161547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-487e2921-a728-4e74-a2e5-90e6299752c7 req-5d9ff57e-d9fb-4d79-90db-b4d0f098c8ab service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:12.162355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-487e2921-a728-4e74-a2e5-90e6299752c7 req-5d9ff57e-d9fb-4d79-90db-b4d0f098c8ab service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:12.177050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-487e2921-a728-4e74-a2e5-90e6299752c7 req-5d9ff57e-d9fb-4d79-90db-b4d0f098c8ab service nova] Creating event network-changed:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:51:12.182806 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-487e2921-a728-4e74-a2e5-90e6299752c7 req-5d9ff57e-d9fb-4d79-90db-b4d0f098c8ab service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036398 May 24 10:51:12.183561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1290/2161] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:51:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:51:12.941968 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:12.942623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:12.950560 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:12.951004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:12.951751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:12.952179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:13.081447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:13.090472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:13.091982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:13.093086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-049d0f84-083f-4527-a7eb-31e0500672af tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1407 microversion: 2.20 time: 0.155492 May 24 10:51:13.093682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1291/2162] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:12 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1407 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:14.111328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:14.112015 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:14.118543 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:14.118985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:14.119735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:14.120805 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:14.253397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:14.261695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:14.265953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:14.265953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7f616c35-de28-4bba-a851-3107a043b539 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1407 microversion: 2.20 time: 0.156923 May 24 10:51:14.265953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1292/2163] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:14 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1407 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:15.282387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:15.283061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:15.289521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:15.290122 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:15.290885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:15.291323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:15.417722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:15.425603 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:15.427083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:15.428162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c6075098-7721-4695-aeff-89f02dfacf83 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1407 microversion: 2.20 time: 0.149738 May 24 10:51:15.428717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1293/2164] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:15 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1407 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:16.433489 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-54b0b129-81bf-444d-bc1f-e1351c694fa2 req-5704bfda-fd82-4cdd-8bb9-6a158fbe2370 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "name": "network-vif-plugged", "status": "completed", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:16.442397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-54b0b129-81bf-444d-bc1f-e1351c694fa2 req-5704bfda-fd82-4cdd-8bb9-6a158fbe2370 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:16.442839 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-54b0b129-81bf-444d-bc1f-e1351c694fa2 req-5704bfda-fd82-4cdd-8bb9-6a158fbe2370 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:16.454506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-54b0b129-81bf-444d-bc1f-e1351c694fa2 req-5704bfda-fd82-4cdd-8bb9-6a158fbe2370 service nova] Creating event network-vif-plugged:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:51:16.459861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-54b0b129-81bf-444d-bc1f-e1351c694fa2 req-5704bfda-fd82-4cdd-8bb9-6a158fbe2370 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030307 May 24 10:51:16.460526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1294/2165] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:51:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:51:16.466763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:16.467456 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:16.473824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:16.474281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:16.475049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:16.475488 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:16.609418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:16.617565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:16.619052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:16.620688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-88b0d9ff-5c1d-4c7d-9927-7ba065504cd4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1407 microversion: 2.20 time: 0.158949 May 24 10:51:16.621330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1295/2166] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:16 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1407 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:51:17.639086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:17.639827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:17.646580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:17.647030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:17.647788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:17.648392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:17.778616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:17.786428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:17.787911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:17.789001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fd350612-ae79-4bd5-81ee-5dbba47e1bb8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1407 microversion: 2.20 time: 0.154012 May 24 10:51:17.789589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1296/2167] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:17 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1407 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:18.807021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:18.807686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:18.809598 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:18.810327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=12 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:18.815577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:18.815994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:18.816752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:18.817183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:18.944759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:18.952922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:18.954381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:18.956159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5ca2dd34-37e3-4fac-9b08-3d3990efc9b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1599 microversion: 2.20 time: 0.152567 May 24 10:51:18.956159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1297/2168] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:18 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1599 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:51:18.973962 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:18.974442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:18.980974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:18.981451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:18.982210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:18.982636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:19.062081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ab0df91c-7d28-4c18-a52a-a7e82929b711 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.091707 May 24 10:51:19.062737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1298/2169] 10.222.0.35 () {60 vars in 1315 bytes} [Tue May 24 10:51:18 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:51:20.275183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-581ef1a0-8dd2-4a3b-8e36-38ea7f596249 req-27eddc71-f3db-4277-a10a-a5f61399d942 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:20.283977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-581ef1a0-8dd2-4a3b-8e36-38ea7f596249 req-27eddc71-f3db-4277-a10a-a5f61399d942 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:20.284418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-581ef1a0-8dd2-4a3b-8e36-38ea7f596249 req-27eddc71-f3db-4277-a10a-a5f61399d942 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:20.298266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-581ef1a0-8dd2-4a3b-8e36-38ea7f596249 req-27eddc71-f3db-4277-a10a-a5f61399d942 service nova] Creating event network-changed:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:51:20.303159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-581ef1a0-8dd2-4a3b-8e36-38ea7f596249 req-27eddc71-f3db-4277-a10a-a5f61399d942 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032230 May 24 10:51:20.303758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1299/2170] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:51:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:51:56.488749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=90,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:56.489525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=46 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:56.494182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:56.494756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:56.500815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:56.508804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:56.508804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:56.508804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:56.525343 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4f9bfdc6-0b94-4680-96e8-84e2d57281a2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/ips" status: 200 len: 152 microversion: 2.20 time: 0.038244 May 24 10:51:56.525921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1300/2171] 10.222.0.35 () {60 vars in 1288 bytes} [Tue May 24 10:51:56 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/ips => generated 152 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:51:58.743009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:51:58.743697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:58.751554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:58.751999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:58.752761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:58.753197 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:58.776278 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lazy-loading 'flavor' on Instance uuid 6f04105c-0517-4050-a90a-3ccfb7c6cd93 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:51:59.295183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6968b5c1-6792-430c-84f8-a88ae07edb01 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/action" status: 202 len: 0 microversion: 2.20 time: 0.556338 May 24 10:51:59.295803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1301/2172] 10.222.0.35 () {62 vars in 1318 bytes} [Tue May 24 10:51:58 2022] POST /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/action => generated 0 bytes in 558 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 24 10:51:59.303813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:51:59.304435 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:51:59.311581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:59.312028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:59.312787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:51:59.313291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:51:59.444050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:59.453076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:59.467932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:51:59.467932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2b27b0d8-4efd-4c90-81c9-335e5704d7cb tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.155887 May 24 10:51:59.467932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1302/2173] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:51:59 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:00.474057 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:00.474783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:00.481550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:00.486052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:00.486052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:00.486052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:00.612421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:00.620997 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:00.622677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:00.623829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9ea010c8-495c-49bb-be86-bff47451e3ae tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.153679 May 24 10:52:00.624390 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1303/2174] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:00 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:01.641970 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:01.642657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:01.649049 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:01.649524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:01.650274 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:01.651182 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:01.782807 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:01.790724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:01.792973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:01.794158 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a198232c-69d3-4212-ba90-f583446e8121 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.156065 May 24 10:52:01.794728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1304/2175] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:01 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:02.812867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:02.813575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:02.820088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:02.820540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:02.821325 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:02.821761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:02.950758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:02.959128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:02.960647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:02.961842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7751c4d0-4935-42bd-88c3-af5eaeb5b74c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.152901 May 24 10:52:02.962397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1305/2176] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:02 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:03.979986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:03.980654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:03.987070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:03.987524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:03.988279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:03.988710 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:04.117541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:04.126517 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:04.128012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:04.129208 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6f19d3c9-10c8-4aba-bf48-aaaf4c7de4b3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.153312 May 24 10:52:04.129795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1306/2177] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:03 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:05.147501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:05.148178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:05.155086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:05.155536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:05.156295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:05.156730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:05.283353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:05.291209 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:05.292711 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:05.293890 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b1ba1fae-6657-4abc-80f5-eb6e22245d86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.150413 May 24 10:52:05.294460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1307/2178] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:05 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:06.312614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:06.313315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:06.319722 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:06.320169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:06.320927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:06.321392 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:06.448428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:06.456326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:06.457852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:06.459258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-55732ab2-e9bb-4adc-848f-54a5da579ca4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.150558 May 24 10:52:06.459822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1308/2179] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:06 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:52:07.476658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:07.477345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:07.483824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:07.484294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:07.485143 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:07.485601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:07.616800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:07.625404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:07.626922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:07.628092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-496c5839-1386-48fc-8e52-a01331999e86 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.155298 May 24 10:52:07.628653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1309/2180] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:07 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:08.647118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:08.647884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:08.654504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:08.654950 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:08.655738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:08.656168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:08.790811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:08.799382 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:08.802368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:08.802368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b44a438e-4ff3-4175-b7fd-b974854425c7 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.158969 May 24 10:52:08.802620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1310/2181] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:08 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:09.821035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:09.821775 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:09.828802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:09.829243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:09.830033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:09.830459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:09.969994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:09.978008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:09.979719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:09.980886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0e7fdf5f-959a-4185-937e-bab4ddfc964a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.163835 May 24 10:52:09.981491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1311/2182] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:09 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:10.999108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:10.999769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:11.006257 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:11.006718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:11.007448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:11.007873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:11.137287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:11.145307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:11.146795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:11.147939 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-0ebcb58b-a5b4-4c7c-93a7-e574e617a544 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.152759 May 24 10:52:11.148497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1312/2183] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:10 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:12.166695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:12.167298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:12.173772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:12.174214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:12.174991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:12.175417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:12.304927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:12.313372 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:12.314853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:12.315991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9c5077c6-4049-4397-a37f-e15b22c137c5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.153260 May 24 10:52:12.316554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1313/2184] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:12 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:13.335353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:13.336040 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:13.342694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:13.343145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:13.343918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:13.344358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:13.471503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:13.475005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:13.480470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:13.482037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:13.483191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5f07afe3-e170-4788-8a32-08a9906dded0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.152117 May 24 10:52:13.483759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1314/2185] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:13 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:14.501556 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:14.502246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:14.509369 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:14.510431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:14.511345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:14.511778 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:14.639637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:14.647938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:14.649565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:14.650832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-89e4db9a-b774-49e8-93b1-17ddc045bf15 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.153175 May 24 10:52:14.651391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1315/2186] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:14 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:15.669213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:15.669957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:15.676484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:15.676934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:15.677717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:15.678152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:15.808787 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:15.816662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:15.818188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:15.819334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ce642024-4b8e-4c40-b463-1eb970f1e235 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.154176 May 24 10:52:15.819892 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1316/2187] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:15 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:16.838535 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:16.839262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:16.845644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:16.846096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:16.846852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:16.847293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:16.973206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:16.981720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:16.983206 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:16.984339 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-43ee8403-bd2b-43ff-874e-ff4eaf3c3415 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.149816 May 24 10:52:16.984899 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1317/2188] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:16 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:18.003075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:18.003781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:18.011427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:18.011869 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:18.012912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:18.013581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:18.146760 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:18.154861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:18.156376 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:18.157628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-99f914d1-b753-44e9-9335-b1f89dd5fc34 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.158422 May 24 10:52:18.158199 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1318/2189] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:17 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:52:19.175956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:19.176544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:19.183904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:19.184354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:19.185114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:19.185575 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:19.314165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:19.322186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:19.323680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:19.324809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6638ce00-fc5f-41b7-b32c-3543a3de09de tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.152783 May 24 10:52:19.325394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1319/2190] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:19 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:20.343911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:20.344541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:20.350860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:20.351306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:20.352054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:20.352486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:20.481280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:20.489350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:20.490836 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:20.491981 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a01ba41d-dacb-4c49-9e7c-c7ce4c71888b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1745 microversion: 2.20 time: 0.152265 May 24 10:52:20.492544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1320/2191] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:20 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1745 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:21.517130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:21.517833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:21.524231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:21.524674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:21.526262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:21.526696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:21.657589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:21.665417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:21.666945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:21.668092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1cc97613-a7cf-4e3e-a3e3-d3702bf45692 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1745 microversion: 2.20 time: 0.154979 May 24 10:52:21.668646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1321/2192] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:21 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1745 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:22.649457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-5606556f-c497-4a5f-b734-efb88930332e req-76164766-47a7-44b9-a09c-47fc3406b76e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "name": "network-vif-plugged", "status": "completed", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:52:22.658217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5606556f-c497-4a5f-b734-efb88930332e req-76164766-47a7-44b9-a09c-47fc3406b76e service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:22.658663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5606556f-c497-4a5f-b734-efb88930332e req-76164766-47a7-44b9-a09c-47fc3406b76e service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:22.670088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-5606556f-c497-4a5f-b734-efb88930332e req-76164766-47a7-44b9-a09c-47fc3406b76e service nova] Creating event network-vif-plugged:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:52:22.675792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-5606556f-c497-4a5f-b734-efb88930332e req-76164766-47a7-44b9-a09c-47fc3406b76e service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030153 May 24 10:52:22.676374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1322/2193] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:52:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:52:22.686120 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:22.689043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:22.692977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:22.693457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:22.694239 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:22.694669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:22.829813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:22.838640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:22.840272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:22.841476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6ce03d01-c767-425d-b409-ebc1fe89b781 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.159216 May 24 10:52:22.842037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1323/2194] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:22 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:23.860764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:23.862401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:23.870032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:23.870788 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:23.871770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:23.872411 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:24.027466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:24.036352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:24.037880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:24.039035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-21a0659f-2508-4b16-97fd-e230974eb0f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.182855 May 24 10:52:24.039978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1324/2195] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:23 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:25.058318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:25.059043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:25.065952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:25.066413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:25.067188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:25.067626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:25.214026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:25.222387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:25.223918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:25.225080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d45d11d4-af73-4928-b4ad-61571719f0a0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.171048 May 24 10:52:25.225676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1325/2196] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:25 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:52:26.248300 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:26.249732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:26.255827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:26.256501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:26.257664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:26.258176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:26.417409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:26.425841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:26.427333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:26.428494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fae5b55d-3f79-459c-9757-008006378bed tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.186038 May 24 10:52:26.429793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1326/2197] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:26 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:52:27.447162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:27.448290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:27.455144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:27.455600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:27.456368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:27.456801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:27.603799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:27.612687 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:27.614212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:27.615375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b74ce132-5ea1-4e96-b902-f9cc954eef17 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.172703 May 24 10:52:27.616265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1327/2198] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:27 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:28.635345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:28.636097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:28.642585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:28.643033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:28.643790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:28.644224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:28.785438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:28.785438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:28.785714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:28.787038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-11e379da-6f87-400a-999c-630eea3a324c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.156350 May 24 10:52:28.787700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1328/2199] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:28 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:29.807359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:29.808019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:29.810717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=59 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:29.815887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:29.816600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:29.817837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:29.822071 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:29.972421 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:29.981705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:29.983461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:29.984714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4307eda2-973a-492d-8fd4-90e568e8d89b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.181705 May 24 10:52:29.985337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1329/2200] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:29 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:52:31.003577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:31.004328 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:31.011911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:31.012353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:31.013121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:31.013577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:31.153809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:31.161814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:31.163306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:31.164460 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bc36b601-d3e2-4f7d-9417-00100c6a1fb2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.164853 May 24 10:52:31.167401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1330/2201] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:30 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:32.186773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:32.187537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:32.194424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:32.194871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:32.195669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:32.196104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:32.330741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:32.339898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:32.341870 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:32.343609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d9700351-f29d-4f3b-a9b4-aed807ada2c2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.165891 May 24 10:52:32.344667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1331/2202] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:32 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:33.362144 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:33.362752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:33.370030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:33.370507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:33.371356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:33.371791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:33.375781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:33.506700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:33.515224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:33.516716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:33.517922 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ac834eea-6437-45d9-b72c-728b44aedb64 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1740 microversion: 2.20 time: 0.160009 May 24 10:52:33.518483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1332/2203] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:33 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1740 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:34.537100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:34.537789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:34.544186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:34.544624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:34.545405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:34.545898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:34.671866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:34.679622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:34.681141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:34.683025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f7d15c75-c1ab-4ccc-97ce-46517f1dfd40 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1722 microversion: 2.20 time: 0.149791 May 24 10:52:34.683591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1333/2204] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:34 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:35.701567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:35.702175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:35.708577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:35.709027 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:35.709804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:35.710233 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:35.839493 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:35.847565 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:35.849126 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:35.850307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-bb354275-3c62-4a81-bbf7-ca2666600629 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1722 microversion: 2.20 time: 0.152675 May 24 10:52:35.850916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1334/2205] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:35 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1722 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:36.649852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-68682ebd-b0ce-4f32-8add-748020f6de9e req-98c893c7-5707-4802-a838-5f391d357af9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "name": "network-vif-unplugged", "status": "completed", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:52:36.658814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-68682ebd-b0ce-4f32-8add-748020f6de9e req-98c893c7-5707-4802-a838-5f391d357af9 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:36.659248 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-68682ebd-b0ce-4f32-8add-748020f6de9e req-98c893c7-5707-4802-a838-5f391d357af9 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:36.670321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-68682ebd-b0ce-4f32-8add-748020f6de9e req-98c893c7-5707-4802-a838-5f391d357af9 service nova] Creating event network-vif-unplugged:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:52:36.675497 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-68682ebd-b0ce-4f32-8add-748020f6de9e req-98c893c7-5707-4802-a838-5f391d357af9 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029398 May 24 10:52:36.676263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1335/2206] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:52:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:52:36.871119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:36.871757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:36.878471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:36.878923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:36.879715 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:36.880149 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:37.014309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:37.022180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:37.023662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:37.024801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-91920989-6410-4a8c-9018-5d1d2d2a3959 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1725 microversion: 2.20 time: 0.158929 May 24 10:52:37.025424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1336/2207] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:36 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1725 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:37.042345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "22a75d72-7f37-40eb-b0f3-0992c0d5aad1"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:52:37.043386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:37.049593 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:37.050045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:37.050833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:37.051227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:37.952474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-749ef296-db19-4e1c-b88a-b6befabbcd85 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.913955 May 24 10:52:37.953150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1337/2208] 10.222.0.35 () {62 vars in 1363 bytes} [Tue May 24 10:52:37 2022] POST /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-volume_attachments => generated 188 bytes in 915 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) May 24 10:52:38.068877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:38.069385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:38.075674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:38.076127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:38.076879 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:38.077337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:38.636064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5c58db4e-d152-41fe-b016-99b52a20bc2a tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-volume_attachments/22a75d72-7f37-40eb-b0f3-0992c0d5aad1" status: 202 len: 0 microversion: 2.20 time: 0.571015 May 24 10:52:38.636591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1338/2209] 10.222.0.35 () {60 vars in 1456 bytes} [Tue May 24 10:52:38 2022] DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-volume_attachments/22a75d72-7f37-40eb-b0f3-0992c0d5aad1 => generated 0 bytes in 572 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 24 10:52:38.692181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:52:38.692783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:38.699685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:38.700138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:38.700894 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:38.701416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:38.909569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-efbc1cfb-0c61-41a9-b57e-05719f8128ad tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "POST /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/action" status: 202 len: 0 microversion: 2.20 time: 0.221132 May 24 10:52:38.910140 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1339/2210] 10.222.0.35 () {62 vars in 1318 bytes} [Tue May 24 10:52:38 2022] POST /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/action => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) May 24 10:52:38.918033 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:38.918639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:38.924669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:38.925155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:38.925988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:38.926433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:39.060794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:39.068895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:39.070467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:39.071684 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c7e0a034-aafb-480a-bb93-5f438ccc7ab0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1673 microversion: 2.20 time: 0.157460 May 24 10:52:39.072326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1340/2211] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:38 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1673 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:52:40.091796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:40.092511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:40.099019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:40.099463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:40.100216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:40.100642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:40.232228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:40.240227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:40.241790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:40.242948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-46d7680b-b842-4daf-bfdd-3be8df47c8b0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1675 microversion: 2.20 time: 0.155936 May 24 10:52:40.243504 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1341/2212] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:40 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1675 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:41.261917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:41.263308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:41.270212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:41.270649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:41.271417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:41.272048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:41.406804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:41.414871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:41.417098 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:41.418266 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-43336b3f-4011-4cd2-8105-479bdce3bcc5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.160440 May 24 10:52:41.418852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1342/2213] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:41 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:42.437028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:42.437721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:42.444179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:42.444624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:42.445408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:42.445841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:42.463391 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=85,nova_cell1:INSERT=6,nova_cell1:UPDATE=9 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:42.584792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:42.593370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:42.594880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:42.596037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-64ebeb9f-e6d1-449f-a4f3-cb1bc785b982 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.163242 May 24 10:52:42.596600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1343/2214] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:42 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:43.613963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:43.614850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:43.623671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:43.624609 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:43.625967 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:43.626791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:43.770841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:43.779364 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:43.780961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:43.782258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cbf7969a-698e-4ea2-bc47-9f10d31d17d4 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.172276 May 24 10:52:43.782833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1344/2215] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:43 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:44.802366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:44.803051 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:44.810418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:44.810864 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:44.811737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:44.812176 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:44.957129 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:44.967827 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:44.968671 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:44.970061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6d4c885e-b0c1-4bba-8b2d-5629ff505b82 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.171574 May 24 10:52:44.970814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1345/2216] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:44 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:45.989355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:45.990175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:45.997539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:45.998225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:45.999004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:45.999705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:46.133235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:46.146589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:46.148186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:46.149455 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1e59503f-2546-4cf0-a39a-274d95e1aad0 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.164074 May 24 10:52:46.150301 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1346/2217] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:45 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:47.169050 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:47.169701 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:47.176703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:47.177243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:47.178150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:47.178578 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:47.333067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:47.341077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:47.342604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:47.343750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b8437f62-daec-4a4d-bd26-660c89e6d02d tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.179746 May 24 10:52:47.344315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1347/2218] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:47 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:48.362580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:48.363492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:48.372409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:48.373023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:48.373907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:48.374406 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:48.519023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:48.527490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:48.529076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:48.530322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5528ffda-70fb-42f9-a5b2-607aa4f15f4b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.171603 May 24 10:52:48.530874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1348/2219] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:48 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:49.548403 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:49.549106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:49.555643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:49.556091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:49.556851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:49.557462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:49.685203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:49.693138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:49.694793 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:49.695957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-193ddd96-adab-43ba-bb6f-86b4d29b5774 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.151426 May 24 10:52:49.696509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1349/2220] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:49 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:50.714418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:50.715237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:50.721688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:50.722138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:50.722901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:50.723336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:50.854097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:50.862498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:50.863977 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:50.865139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e1eced83-b42c-4786-8be4-f13672135de8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.154773 May 24 10:52:50.865725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1350/2221] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:50 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:51.884437 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:51.885069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:51.892285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:51.892733 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:51.893521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:51.893957 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:52.021483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:52.029487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:52.030974 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:52.032161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-be9dfcce-5528-4053-a5e3-1ec898cd33ab tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.151551 May 24 10:52:52.032667 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1351/2222] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:51 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:53.049809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:53.050447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:53.056878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:53.057360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:53.058114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:53.059165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:53.191358 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:53.199407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:53.201077 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:53.202296 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-26341d7d-5e7f-4e36-a4f7-9b710eeea628 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.156415 May 24 10:52:53.202841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1352/2223] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:53 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:54.220621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:54.221238 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:54.227814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:54.228265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:54.229020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:54.229503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:54.358909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:54.366812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:54.368286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:54.369454 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ecc7632b-72be-427e-9852-67139f35b138 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.152811 May 24 10:52:54.370008 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1353/2224] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:54 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:54.808336 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-76d7031d-25a8-452d-9d9c-b6e6209bc85c req-d3809305-50e9-4d26-8c92-0462928732c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "name": "network-vif-plugged", "status": "completed", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:52:54.821175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-76d7031d-25a8-452d-9d9c-b6e6209bc85c req-d3809305-50e9-4d26-8c92-0462928732c5 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:54.821175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-76d7031d-25a8-452d-9d9c-b6e6209bc85c req-d3809305-50e9-4d26-8c92-0462928732c5 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:54.829755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-76d7031d-25a8-452d-9d9c-b6e6209bc85c req-d3809305-50e9-4d26-8c92-0462928732c5 service nova] Creating event network-vif-plugged:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:52:54.835833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-76d7031d-25a8-452d-9d9c-b6e6209bc85c req-d3809305-50e9-4d26-8c92-0462928732c5 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031037 May 24 10:52:54.836461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1354/2225] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:52:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:52:55.388953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:55.389673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:55.395875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:55.399574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:55.399574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:55.399574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:55.542333 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:55.551381 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:55.552895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:55.554092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-eb4050c7-3cf3-4532-9f2c-c94dae6194f8 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.170411 May 24 10:52:55.554900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1355/2226] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:55 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:56.572038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:56.572730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:56.579568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:56.580026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:56.580790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:56.581225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:56.706954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:56.715181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:56.716641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:56.717815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a900ef23-90fc-456c-9ca0-fd808d34b974 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.149743 May 24 10:52:56.718370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1356/2227] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:56 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:57.736462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:57.737110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:57.743721 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:57.744169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:57.744931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:57.745393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:57.871700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:57.879682 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:57.881185 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:57.882365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7375caa7-4cbf-476f-bf1d-ec9d4775866b tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1731 microversion: 2.20 time: 0.149870 May 24 10:52:57.882931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1357/2228] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:57 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1731 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:52:58.899949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:52:58.900601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:52:58.907095 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:58.907544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:58.908302 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:52:58.908740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:52:59.038903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:59.047756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:59.049235 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:52:59.050395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-14cc17b1-f90e-4657-a2b5-d98fcdd85225 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1718 microversion: 2.20 time: 0.154470 May 24 10:52:59.050960 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1358/2229] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:52:58 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1718 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:53:31.958409 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:31.959297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:31.961521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=61 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:31.962162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=89 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:31.967492 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:31.967930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:31.968681 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:31.969108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:31.999036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b28e2a95-e7af-4570-b9ec-21cc63bb862f tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.045011 May 24 10:53:31.999587 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1359/2230] 10.222.0.35 () {60 vars in 1342 bytes} [Tue May 24 10:53:31 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:53:32.058181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:32.058673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:32.066312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:32.066759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:32.067507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:32.067940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:32.092361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:53:32.343394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5fd1b086-4dc9-4166-aaaa-7e3e8b837300 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 204 len: 0 microversion: 2.20 time: 0.287783 May 24 10:53:32.343937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1360/2231] 10.222.0.35 () {60 vars in 1279 bytes} [Tue May 24 10:53:32 2022] DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 24 10:53:32.352415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:32.353193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:32.360846 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:32.361326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:32.362286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:32.362725 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:32.544548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:32.552885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:32.554388 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:32.555524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d4fc6476-a7d6-4195-8d25-e046a4d97dd3 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.207192 May 24 10:53:32.556087 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1361/2232] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:32 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:53:33.575309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:33.575998 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:33.583412 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:33.583854 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:33.584600 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:33.585032 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:33.712573 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:33.721433 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:33.722947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:33.724092 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-36a32130-9a1e-4ffa-a50c-0894c3f365e5 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1724 microversion: 2.20 time: 0.152728 May 24 10:53:33.724657 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1362/2233] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:33 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1724 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:53:34.742627 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:34.743244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:34.749852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:34.750394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:34.751245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:34.751756 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:34.814736 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:34.822900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:34.824422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:34.825612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-12a63c4c-cf67-4c78-9186-8b6f0435489e tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1657 microversion: 2.20 time: 0.087134 May 24 10:53:34.826183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1363/2234] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:34 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1657 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) May 24 10:53:34.901581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-484cc52c-dd97-4312-95b5-5e0abe7b0912 req-58a36cd2-762a-454f-831e-3344e9017145 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6f04105c-0517-4050-a90a-3ccfb7c6cd93", "name": "network-vif-deleted", "tag": "720e7202-574b-46d4-9771-179f82863e86"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:53:34.910743 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-484cc52c-dd97-4312-95b5-5e0abe7b0912 req-58a36cd2-762a-454f-831e-3344e9017145 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:34.911344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-484cc52c-dd97-4312-95b5-5e0abe7b0912 req-58a36cd2-762a-454f-831e-3344e9017145 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:34.924813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-484cc52c-dd97-4312-95b5-5e0abe7b0912 req-58a36cd2-762a-454f-831e-3344e9017145 service nova] Creating event network-vif-deleted:720e7202-574b-46d4-9771-179f82863e86 for instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 on hv-cinder-84311 May 24 10:53:34.930397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-484cc52c-dd97-4312-95b5-5e0abe7b0912 req-58a36cd2-762a-454f-831e-3344e9017145 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102470 May 24 10:53:34.931461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1364/2235] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:53:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:53:35.846661 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:35.847357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:35.854626 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:35.855062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:35.855867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:35.856281 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:35.888993 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:35.889673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:35.890125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lazy-loading 'fault' on Instance uuid 6f04105c-0517-4050-a90a-3ccfb7c6cd93 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:53:35.931108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:35.938420 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:35.939910 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0e46c23c8e3946238795b800b921380c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '107c9dab6bd24705b50f8d30181d6582', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:35.941057 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9c81fdab-aad9-425f-95a5-a46a7c9285a1 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 200 len: 1372 microversion: 2.20 time: 0.099900 May 24 10:53:35.941660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1365/2236] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:35 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 1372 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) May 24 10:53:36.959360 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:36.960062 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:36.966379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:36.966821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:36.968377 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:36.968816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:36.981744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] HTTP exception thrown: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Request method failure captured: May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 HTTP/1.1 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLf24EQxpzEOxG0lwZ9oMlMGcLBXaR2jD6i-8F8R384A4Vuo1B6Nl8m6_z52Qw8ILr6ht3fR1X-fKF-LungyjKRkkGr-e2OgIIMZ6PUJ5-MNYpr1N2muUGkraoEydMWJ-7bQWpApKakXOe7XslJU301k65t3bTcC_SHe4dF8jpN84GA May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Openstack-Nova-Api-Version: 2.20 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:53:36.983620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 0e46c23c8e3946238795b800b921380c May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:36.985596 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '6f04105c-0517-4050-a90a-3ccfb7c6cd93'} May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:53:36.986653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:36.988357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:36.989930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Returning 404 to user: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:53:36.989930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f346bfaf-76f8-49a2-8395-738e0d207e0c tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 404 len: 111 microversion: 2.20 time: 0.029772 May 24 10:53:36.989930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1366/2237] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:36 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 24 10:53:36.993335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-88c38be3-f03b-42f0-a44a-df23764f5cea tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '.version_select at 0x7f8737512940>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:36.996434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR oslo.messaging._drivers.impl_rabbit [None req-88c38be3-f03b-42f0-a44a-df23764f5cea tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [6047ad6b-22c1-4aaa-9f61-adaa8777c5ab] AMQP server on 10.222.0.35:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer May 24 10:53:38.003785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [None req-88c38be3-f03b-42f0-a44a-df23764f5cea tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [6047ad6b-22c1-4aaa-9f61-adaa8777c5ab] Reconnected to AMQP server on 10.222.0.35:5672 via [amqp] client with port 52580. May 24 10:53:38.112263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-88c38be3-f03b-42f0-a44a-df23764f5cea tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1336007704" status: 204 len: 0 microversion: 2.20 time: 1.122251 May 24 10:53:38.112844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1367/2238] 10.222.0.35 () {60 vars in 1261 bytes} [Tue May 24 10:53:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1336007704 => generated 0 bytes in 1123 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) May 24 10:53:40.370628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:40.371218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:40.380250 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:40.380699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:40.381494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:40.381934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:40.394891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] HTTP exception thrown: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Request method failure captured: May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 HTTP/1.1 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLf24EQxpzEOxG0lwZ9oMlMGcLBXaR2jD6i-8F8R384A4Vuo1B6Nl8m6_z52Qw8ILr6ht3fR1X-fKF-LungyjKRkkGr-e2OgIIMZ6PUJ5-MNYpr1N2muUGkraoEydMWJ-7bQWpApKakXOe7XslJU301k65t3bTcC_SHe4dF8jpN84GA May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Openstack-Nova-Api-Version: 2.20 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:53:40.396471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 0e46c23c8e3946238795b800b921380c May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.398269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '6f04105c-0517-4050-a90a-3ccfb7c6cd93'} May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:53:40.399244 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.400618 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.402036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Returning 404 to user: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:53:40.402036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9943a6cc-b8f4-4ff3-abb4-8260a12c1fbd tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 404 len: 111 microversion: 2.20 time: 0.031618 May 24 10:53:40.402036 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1368/2239] 10.222.0.35 () {60 vars in 1279 bytes} [Tue May 24 10:53:40 2022] DELETE /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 24 10:53:40.406352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:40.406949 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: 6f04105c-0517-4050-a90a-3ccfb7c6cd93] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:40.412624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:40.413072 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:40.413872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:40.414311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:40.427658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] HTTP exception thrown: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Request method failure captured: May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 HTTP/1.1 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLf24EQxpzEOxG0lwZ9oMlMGcLBXaR2jD6i-8F8R384A4Vuo1B6Nl8m6_z52Qw8ILr6ht3fR1X-fKF-LungyjKRkkGr-e2OgIIMZ6PUJ5-MNYpr1N2muUGkraoEydMWJ-7bQWpApKakXOe7XslJU301k65t3bTcC_SHe4dF8jpN84GA May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Openstack-Nova-Api-Version: 2.20 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:53:40.429136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 0e46c23c8e3946238795b800b921380c May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.430868 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '6f04105c-0517-4050-a90a-3ccfb7c6cd93'} May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:53:40.431695 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. May 24 10:53:40.433075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:40.434371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Returning 404 to user: Instance 6f04105c-0517-4050-a90a-3ccfb7c6cd93 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:53:40.434371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e59d6b8a-cfa4-404a-87ca-740bab275211 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93" status: 404 len: 111 microversion: 2.20 time: 0.028245 May 24 10:53:40.434371 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1369/2240] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:40 2022] GET /compute/v2.1/servers/6f04105c-0517-4050-a90a-3ccfb7c6cd93 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 24 10:53:41.696856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:41.697475 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:41.703767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:41.704207 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:41.704955 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:41.705416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:41.719735 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] HTTP exception thrown: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Request method failure captured: May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 HTTP/1.1 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLf24EQxpzEOxG0lwZ9oMlMGcLBXaR2jD6i-8F8R384A4Vuo1B6Nl8m6_z52Qw8ILr6ht3fR1X-fKF-LungyjKRkkGr-e2OgIIMZ6PUJ5-MNYpr1N2muUGkraoEydMWJ-7bQWpApKakXOe7XslJU301k65t3bTcC_SHe4dF8jpN84GA May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Openstack-Nova-Api-Version: 2.20 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:53:41.721317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 0e46c23c8e3946238795b800b921380c May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.723139 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'ac4af8cb-8880-4a4e-84d9-b4170d85e4a4'} May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:53:41.724386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.726213 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.727824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Returning 404 to user: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:53:41.727824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a383c5c8-bd86-4afd-b3be-558139950de2 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 404 len: 111 microversion: 2.20 time: 0.029615 May 24 10:53:41.727824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1370/2241] 10.222.0.35 () {60 vars in 1279 bytes} [Tue May 24 10:53:41 2022] DELETE /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) May 24 10:53:41.730548 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:41.731148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] [instance: ac4af8cb-8880-4a4e-84d9-b4170d85e4a4] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:41.737643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:41.738080 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:41.738853 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:41.739289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:41.753085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] HTTP exception thrown: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Request method failure captured: May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 HTTP/1.1 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLf24EQxpzEOxG0lwZ9oMlMGcLBXaR2jD6i-8F8R384A4Vuo1B6Nl8m6_z52Qw8ILr6ht3fR1X-fKF-LungyjKRkkGr-e2OgIIMZ6PUJ5-MNYpr1N2muUGkraoEydMWJ-7bQWpApKakXOe7XslJU301k65t3bTcC_SHe4dF8jpN84GA May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Openstack-Nova-Api-Version: 2.20 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:53:41.754632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/107c9dab6bd24705b50f8d30181d6582"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: 107c9dab6bd24705b50f8d30181d6582 May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1555930894 May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: 0e46c23c8e3946238795b800b921380c May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1555930894-project May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.756470 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'ac4af8cb-8880-4a4e-84d9-b4170d85e4a4'} May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:53:41.757495 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. May 24 10:53:41.759136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:53:41.760676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] Returning 404 to user: Instance ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:53:41.760676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a4356ac0-c1b2-4d0f-9447-6f8a15666219 tempest-AttachVolumeShelveTestJSON-1555930894 tempest-AttachVolumeShelveTestJSON-1555930894-project] 10.222.0.35 "GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4" status: 404 len: 111 microversion: 2.20 time: 0.028945 May 24 10:53:41.760676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1371/2242] 10.222.0.35 () {60 vars in 1276 bytes} [Tue May 24 10:53:41 2022] GET /compute/v2.1/servers/ac4af8cb-8880-4a4e-84d9-b4170d85e4a4 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) May 24 10:53:52.357385 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:52.359948 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:52.476866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7a08594d-bda2-455b-a37b-1692d54ba14e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: .version_select at 0x7f8733761af0>, body: {"keypair": {"name": "tempest-keypair-610908888", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAxP5KHtqChaQBQFnty/hZmmXZWs8fFUYBal9GDsu1i8uX1oKpICs4k/wnCyw5uQRPowxsiP1bK7TUFipcgr8bwcC6fLqm1YfokDWqroynd0nb/9c4G+I8ITLl5wXpbfjA=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:53:52.493699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-7a08594d-bda2-455b-a37b-1692d54ba14e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:53:52.498752 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-7a08594d-bda2-455b-a37b-1692d54ba14e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:53:52.599245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7a08594d-bda2-455b-a37b-1692d54ba14e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.247341 May 24 10:53:52.603441 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1372/2243] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:53:52 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:53:54.718908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1350593120", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "28f3e00d-1916-440e-8708-49c75b5dbe5a"}], "security_groups": [{"name": "tempest-securitygroup--2113767713"}], "key_name": "tempest-keypair-610908888", "user_data": "}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:53:54.821415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] validate_networks() for [('28f3e00d-1916-440e-8708-49c75b5dbe5a', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:53:55.281712 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:53:55.282145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:53:55.282619 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:53:55.283026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:53:55.283434 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:53:55.292781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:55.293874 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:55.294688 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:55.295130 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:55.317852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:53:55.322665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:53:55.335698 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:53:55.338223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:53:55.484427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-85b6510c-b0c5-43ed-987d-ecc20bf02949 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.770028 May 24 10:53:55.485362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1373/2244] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:53:54 2022] POST /compute/v2.1/servers => generated 402 bytes in 772 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) May 24 10:53:55.495224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ba3ebc07-76c1-4a9d-85a5-5978ee88281d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:55.501164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ba3ebc07-76c1-4a9d-85a5-5978ee88281d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:55.562658 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-ba3ebc07-76c1-4a9d-85a5-5978ee88281d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:55.572432 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ba3ebc07-76c1-4a9d-85a5-5978ee88281d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1241 microversion: 2.1 time: 0.081243 May 24 10:53:55.575464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1374/2245] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:53:55 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1241 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:53:56.590903 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:56.591574 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:56.598183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:56.598630 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:56.599393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:56.603127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:56.663223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:56.676643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5d624b04-be0d-40e5-a28e-5c149f6c4aaa tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1245 microversion: 2.1 time: 0.089444 May 24 10:53:56.683438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1375/2246] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:53:56 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1245 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:53:57.696629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:57.697368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:57.703416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:57.703828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:57.704586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:57.705019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:57.767081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:57.774741 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2031134c-fb4d-4910-83d4-389f5df13747 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1301 microversion: 2.1 time: 0.082085 May 24 10:53:57.775316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1376/2247] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:53:57 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1301 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:53:58.792162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:58.792887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:58.801154 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:58.804652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:58.804652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:58.804652 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:58.947429 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:53:58.955883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e48073aa-2835-48a4-a6d4-ec68364e2d5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.167673 May 24 10:53:58.956463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1377/2248] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:53:58 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:53:59.714466 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-0df5c5f1-9776-4616-b8a1-fd3597d5d14a req-d8c2bc7a-fec8-4619-9e9a-b5e31a85d841 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9405c433-2f5d-4f00-86f5-6b632883f1c6", "tag": "17e449ec-23f4-46ba-bf62-0a0bb01b298f"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:53:59.725690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-0df5c5f1-9776-4616-b8a1-fd3597d5d14a req-d8c2bc7a-fec8-4619-9e9a-b5e31a85d841 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:59.726472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-0df5c5f1-9776-4616-b8a1-fd3597d5d14a req-d8c2bc7a-fec8-4619-9e9a-b5e31a85d841 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:59.739789 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-0df5c5f1-9776-4616-b8a1-fd3597d5d14a req-d8c2bc7a-fec8-4619-9e9a-b5e31a85d841 service nova] Creating event network-changed:17e449ec-23f4-46ba-bf62-0a0bb01b298f for instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 on hv-cinder-84311 May 24 10:53:59.745175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-0df5c5f1-9776-4616-b8a1-fd3597d5d14a req-d8c2bc7a-fec8-4619-9e9a-b5e31a85d841 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039200 May 24 10:53:59.748501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1378/2249] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:53:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:53:59.975231 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:53:59.975871 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:53:59.981984 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:59.982430 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:53:59.983832 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:53:59.984269 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:00.114761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:00.122332 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-de3a0b6b-873c-478d-968d-67d78d112e33 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.151776 May 24 10:54:00.122909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1379/2250] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:53:59 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:01.139461 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:01.140263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:01.146660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:01.147108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:01.147873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:01.148309 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:01.280644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:01.288086 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8f4eb783-8fd3-4689-900c-7800eb63147d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.152717 May 24 10:54:01.288654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1380/2251] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:01 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:02.305423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:02.306059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:02.312659 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:02.313105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:02.313885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:02.314316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:02.440275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:02.447703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dc5a803b-89ee-47f9-ba98-0e6e26b533c0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.146116 May 24 10:54:02.448279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1381/2252] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:02 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:03.465256 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:03.465912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:03.472770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:03.473221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:03.474046 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:03.474483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:03.602666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:03.611096 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1a102606-755e-4c20-aecd-5d3ed86fa445 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.149637 May 24 10:54:03.611694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1382/2253] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:03 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:04.628306 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:04.628973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:04.635397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:04.635845 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:04.636601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:04.637035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:04.765724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:04.773055 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-42bbe48f-6bca-408f-acd7-b1d1adf4c092 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.149158 May 24 10:54:04.773651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1383/2254] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:04 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:05.789835 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:05.790498 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:05.792366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:05.793101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=12 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:05.798357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:05.798810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:05.799569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:05.800001 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:05.929099 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:05.937227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5e75a607-5d8a-408e-a844-2e9eaf175cdf tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.151320 May 24 10:54:05.937815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1384/2255] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:05 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:06.954146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:06.954767 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:06.961472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:06.961917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:06.962674 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:06.963106 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:07.096162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:07.103654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-87673147-56d9-4892-87c0-729fa1dc25f1 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.153384 May 24 10:54:07.104230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1385/2256] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:06 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:07.168341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-943a637f-f9c1-4f5d-a9f3-0d50f620c778 req-80e09316-84af-4bfd-b8ac-cda8ca3f0c22 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9405c433-2f5d-4f00-86f5-6b632883f1c6", "name": "network-vif-plugged", "status": "completed", "tag": "17e449ec-23f4-46ba-bf62-0a0bb01b298f"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:54:07.178091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-943a637f-f9c1-4f5d-a9f3-0d50f620c778 req-80e09316-84af-4bfd-b8ac-cda8ca3f0c22 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:07.178934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-943a637f-f9c1-4f5d-a9f3-0d50f620c778 req-80e09316-84af-4bfd-b8ac-cda8ca3f0c22 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:07.191755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-943a637f-f9c1-4f5d-a9f3-0d50f620c778 req-80e09316-84af-4bfd-b8ac-cda8ca3f0c22 service nova] Creating event network-vif-plugged:17e449ec-23f4-46ba-bf62-0a0bb01b298f for instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 on hv-cinder-84311 May 24 10:54:07.197039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-943a637f-f9c1-4f5d-a9f3-0d50f620c778 req-80e09316-84af-4bfd-b8ac-cda8ca3f0c22 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088834 May 24 10:54:07.197615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1386/2257] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:54:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:54:08.120621 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:08.121345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:08.128852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:08.129323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:08.130073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:08.130529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:08.260217 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:08.268509 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-282bfc95-3673-4ba3-a6b2-5469d4f6d9da tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1367 microversion: 2.1 time: 0.152293 May 24 10:54:08.269075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1387/2258] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:08 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1367 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:09.286115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:09.286841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:09.293245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:09.293792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:09.294551 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:09.294980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:09.432919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:09.440714 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-64591ae8-7e38-4c23-bf57-d82ef08a1c21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1557 microversion: 2.1 time: 0.158409 May 24 10:54:09.441308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1388/2259] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:09 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1557 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:54:09.458499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:09.458989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:09.465758 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:09.466313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:09.467090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:09.467529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:09.548586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9632e225-959a-42ad-b883-642b0a8bef39 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094157 May 24 10:54:09.549187 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1389/2260] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:54:09 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:54:10.594029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-25e07448-562d-4850-868a-0c43f69d4055 req-e7d3cddd-ef47-499a-a821-9cfae506115a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9405c433-2f5d-4f00-86f5-6b632883f1c6", "tag": "17e449ec-23f4-46ba-bf62-0a0bb01b298f"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:54:10.602162 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-25e07448-562d-4850-868a-0c43f69d4055 req-e7d3cddd-ef47-499a-a821-9cfae506115a service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:10.602728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-25e07448-562d-4850-868a-0c43f69d4055 req-e7d3cddd-ef47-499a-a821-9cfae506115a service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:10.614759 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-25e07448-562d-4850-868a-0c43f69d4055 req-e7d3cddd-ef47-499a-a821-9cfae506115a service nova] Creating event network-changed:17e449ec-23f4-46ba-bf62-0a0bb01b298f for instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 on hv-cinder-84311 May 24 10:54:10.620041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-25e07448-562d-4850-868a-0c43f69d4055 req-e7d3cddd-ef47-499a-a821-9cfae506115a service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031006 May 24 10:54:10.620632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1390/2261] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:54:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:54:46.750330 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:46.751159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=84 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:46.755227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:46.755799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:46.761856 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:46.768295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:46.768295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:46.768295 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:46.790089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-12342d1a-b4c7-4060-98ca-893191c421f4 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/ips" status: 200 len: 150 microversion: 2.1 time: 0.041713 May 24 10:54:46.790745 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1391/2262] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:54:46 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:54:48.397378 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "506e7d46-e487-4c8b-96c9-e909a582ce64"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:54:48.398447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:48.405413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:48.406361 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:48.407111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:48.407544 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:49.069476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f62c4147-2050-4237-9599-f70a197a2c98 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.676156 May 24 10:54:49.070100 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1392/2263] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:54:48 2022] POST /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments => generated 194 bytes in 678 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:54:56.573786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7dbdc3b8-9a42-4cc3-9bb0-258babcfe57a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: .version_select at 0x7f87374d6700>, body: {"keypair": {"name": "tempest-keypair-471115130", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC//cm0IGl2twMIWF2j16VKV/kLZUwC1bdeDwjKNCCqNXZuUHz8dsrcosCZqJzVj5ijHgQPRx43B8vcm3fzfSN6D3l8jmjF+Ui9vZD3p0PaohV+oarcv2UjS2e9PUrkoPw=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:54:56.587479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-7dbdc3b8-9a42-4cc3-9bb0-258babcfe57a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:54:56.591569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-7dbdc3b8-9a42-4cc3-9bb0-258babcfe57a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:54:56.717749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7dbdc3b8-9a42-4cc3-9bb0-258babcfe57a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.148553 May 24 10:54:56.718272 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1393/2264] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:54:56 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:54:58.795042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1298309076", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "28f3e00d-1916-440e-8708-49c75b5dbe5a"}], "security_groups": [{"name": "tempest-securitygroup--651116232"}], "key_name": "tempest-keypair-471115130", "user_data": "}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:54:58.896921 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] validate_networks() for [('28f3e00d-1916-440e-8708-49c75b5dbe5a', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:54:59.010946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=15,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:59.265384 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:54:59.265829 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:54:59.266819 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:54:59.267221 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:54:59.267624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:54:59.276263 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:59.276702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:59.277523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:54:59.277951 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:54:59.294782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:54:59.298764 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:54:59.309410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:54:59.311706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:54:59.449121 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dccca053-8153-4005-bb93-e61685502453 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.661491 May 24 10:54:59.449810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1394/2265] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:54:58 2022] POST /compute/v2.1/servers => generated 401 bytes in 663 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 24 10:54:59.461862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1a29cb8c-9de5-4aa1-ae39-cb2fa788170f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:54:59.462523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1a29cb8c-9de5-4aa1-ae39-cb2fa788170f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:54:59.518259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1a29cb8c-9de5-4aa1-ae39-cb2fa788170f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:54:59.524590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1a29cb8c-9de5-4aa1-ae39-cb2fa788170f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1241 microversion: 2.1 time: 0.066566 May 24 10:54:59.525156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1395/2266] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:54:59 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1241 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:00.540848 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:00.541622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:00.549311 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:00.549699 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:00.550472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:00.550912 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:00.615524 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:00.623151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5b0def54-1cbf-4b93-97e0-43e2bcd0086d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1245 microversion: 2.1 time: 0.086210 May 24 10:55:00.623746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1396/2267] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:00 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1245 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:01.640791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:01.641702 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:01.648229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:01.648753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:01.649628 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:01.655205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:01.719706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:01.727677 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-929b44a5-da04-492b-9748-d47bdfb5ed34 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1311 microversion: 2.1 time: 0.091300 May 24 10:55:01.728452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1397/2268] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:01 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1311 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:02.745622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:02.746329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:02.753818 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:02.756569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:02.756569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:02.756569 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:02.887245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:02.895994 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b4c3eeb4-3112-4f67-a99a-18ea13c4a4ec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.153721 May 24 10:55:02.905761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1398/2269] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:02 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:03.177340 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-7dc0de5a-f1ee-4756-a407-8daf2a338d0c req-a4c5ee39-a2b2-4634-873c-1fd70828652f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "319b1d20-c508-46d3-b6af-850a23fd0e8c", "tag": "bc3cb441-e98a-4708-90a7-614e612f2219"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:55:03.186529 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7dc0de5a-f1ee-4756-a407-8daf2a338d0c req-a4c5ee39-a2b2-4634-873c-1fd70828652f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:03.187465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7dc0de5a-f1ee-4756-a407-8daf2a338d0c req-a4c5ee39-a2b2-4634-873c-1fd70828652f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:03.199844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-7dc0de5a-f1ee-4756-a407-8daf2a338d0c req-a4c5ee39-a2b2-4634-873c-1fd70828652f service nova] Creating event network-changed:bc3cb441-e98a-4708-90a7-614e612f2219 for instance 319b1d20-c508-46d3-b6af-850a23fd0e8c on hv-cinder-84311 May 24 10:55:03.205113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-7dc0de5a-f1ee-4756-a407-8daf2a338d0c req-a4c5ee39-a2b2-4634-873c-1fd70828652f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035414 May 24 10:55:03.205763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1399/2270] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:55:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:55:03.913347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:03.914039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:03.920262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:03.920708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:03.921494 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:03.921926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:04.053982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:04.061491 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1c3ebae5-0103-4832-acc2-f6404d27d737 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.152723 May 24 10:55:04.062181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1400/2271] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:03 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:05.077645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:05.078344 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:05.085193 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:05.085673 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:05.086428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:05.086861 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:05.214642 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:05.222102 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8f87f921-1623-4690-99f2-a0437e35326e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.148318 May 24 10:55:05.222666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1401/2272] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:05 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:06.239070 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:06.239731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:06.246276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:06.246720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:06.247499 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:06.247929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:06.373451 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:06.381356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-32937bfa-d365-4370-8b53-3f3cbf20939f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.146297 May 24 10:55:06.381898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1402/2273] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:06 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:07.398286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:07.398931 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:07.405444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:07.405901 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:07.407613 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:07.408422 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:07.544321 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:07.551935 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-535c0fc8-d7d0-4157-ad38-1a2d31111c55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.157608 May 24 10:55:07.552503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1403/2274] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:07 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:08.567878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:08.568567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:08.575038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:08.575505 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:08.576316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:08.576770 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:08.705436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:08.713012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4d217276-0008-494c-8e54-f16d3c1665c7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.148987 May 24 10:55:08.713611 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1404/2275] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:08 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:09.416171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-a9d413b8-e989-4ff9-a2ba-4c8f9ddd3afc req-0ef64ffd-b93b-4c6f-8821-d1a3c5f2770c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "319b1d20-c508-46d3-b6af-850a23fd0e8c", "name": "network-vif-plugged", "status": "completed", "tag": "bc3cb441-e98a-4708-90a7-614e612f2219"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:55:09.419828 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:55:09.426262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-a9d413b8-e989-4ff9-a2ba-4c8f9ddd3afc req-0ef64ffd-b93b-4c6f-8821-d1a3c5f2770c service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:09.426704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-a9d413b8-e989-4ff9-a2ba-4c8f9ddd3afc req-0ef64ffd-b93b-4c6f-8821-d1a3c5f2770c service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:09.438401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-a9d413b8-e989-4ff9-a2ba-4c8f9ddd3afc req-0ef64ffd-b93b-4c6f-8821-d1a3c5f2770c service nova] Creating event network-vif-plugged:bc3cb441-e98a-4708-90a7-614e612f2219 for instance 319b1d20-c508-46d3-b6af-850a23fd0e8c on hv-cinder-84311 May 24 10:55:09.443554 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-a9d413b8-e989-4ff9-a2ba-4c8f9ddd3afc req-0ef64ffd-b93b-4c6f-8821-d1a3c5f2770c service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031299 May 24 10:55:09.444166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1405/2276] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:55:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:55:09.728666 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=12 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:55:09.733559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:09.734041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:09.741581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:09.742031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:09.742797 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:09.743225 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:09.891634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:09.899889 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c61aab46-65a9-4a7a-a93d-0ab93eb69c1c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.172149 May 24 10:55:09.903749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1406/2277] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:09 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:10.917037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:10.917763 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:10.925601 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:10.926060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:10.926833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:10.927303 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:11.057201 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:11.065533 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5b51ec7a-3d87-45b1-9d78-611e8a490845 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1366 microversion: 2.1 time: 0.152529 May 24 10:55:11.066108 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1407/2278] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:10 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1366 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:55:12.082521 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:12.083160 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:12.089507 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:12.089958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:12.090716 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:12.091151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:12.220293 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:12.227732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6ad99040-f810-42b8-be13-34cc80142aa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1556 microversion: 2.1 time: 0.149358 May 24 10:55:12.228307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1408/2279] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:12 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1556 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:12.244147 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:12.244629 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:12.250447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:12.250897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:12.251654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:12.252090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:12.329755 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1d652d4-47c4-4d28-ad90-b166eed9ade5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089048 May 24 10:55:12.330353 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1409/2280] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:55:12 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/os-interface => generated 284 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:55:13.375045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-a00ad258-45da-4563-9b6b-5fc2e0265a00 req-9d0b125f-f91a-4900-a599-face511364b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "319b1d20-c508-46d3-b6af-850a23fd0e8c", "tag": "bc3cb441-e98a-4708-90a7-614e612f2219"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:55:13.383260 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-a00ad258-45da-4563-9b6b-5fc2e0265a00 req-9d0b125f-f91a-4900-a599-face511364b3 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:13.383707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-a00ad258-45da-4563-9b6b-5fc2e0265a00 req-9d0b125f-f91a-4900-a599-face511364b3 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:13.395479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-a00ad258-45da-4563-9b6b-5fc2e0265a00 req-9d0b125f-f91a-4900-a599-face511364b3 service nova] Creating event network-changed:bc3cb441-e98a-4708-90a7-614e612f2219 for instance 319b1d20-c508-46d3-b6af-850a23fd0e8c on hv-cinder-84311 May 24 10:55:13.400691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-a00ad258-45da-4563-9b6b-5fc2e0265a00 req-9d0b125f-f91a-4900-a599-face511364b3 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085142 May 24 10:55:13.401561 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1410/2281] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:55:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:55:49.186612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:49.187211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:49.189386 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:55:49.190009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=78 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:55:49.194982 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.195525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.196298 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.197572 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.220811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a834194f-155a-4c22-93f2-33bf0958c22f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/ips" status: 200 len: 150 microversion: 2.1 time: 0.038625 May 24 10:55:49.221511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1411/2282] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:55:49 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/ips => generated 150 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:55:49.231357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "506e7d46-e487-4c8b-96c9-e909a582ce64"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:55:49.232393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:49.238834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.239279 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.240026 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.240452 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.319904 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Invalid volume: volume 506e7d46-e487-4c8b-96c9-e909a582ce64 is already attached to instances: 9405c433-2f5d-4f00-86f5-6b632883f1c6 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: POST /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/os-volume_attachments HTTP/1.1 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 74 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:55:49.322536 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {"volumeAttachment": {"volumeId": "506e7d46-e487-4c8b-96c9-e909a582ce64"}} May 24 10:55:49.324367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Invalid volume: volume 506e7d46-e487-4c8b-96c9-e909a582ce64 is already attached to instances: 9405c433-2f5d-4f00-86f5-6b632883f1c6 May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'server_id': '319b1d20-c508-46d3-b6af-850a23fd0e8c', 'body': {'volumeAttachment': {'volumeId': '506e7d46-e487-4c8b-96c9-e909a582ce64'}}} May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 506e7d46-e487-4c8b-96c9-e909a582ce64 is already attached to instances: 9405c433-2f5d-4f00-86f5-6b632883f1c6 May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:49.325643 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 506e7d46-e487-4c8b-96c9-e909a582ce64 is already attached to instances: 9405c433-2f5d-4f00-86f5-6b632883f1c6 May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:55:49.327582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 506e7d46-e487-4c8b-96c9-e909a582ce64 is already attached to instances: 9405c433-2f5d-4f00-86f5-6b632883f1c6 May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 400 to user: Invalid volume: volume 506e7d46-e487-4c8b-96c9-e909a582ce64 is already attached to instances: 9405c433-2f5d-4f00-86f5-6b632883f1c6 {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f474d84b-5a09-46be-b53f-1dc7d3bb9434 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.096389 May 24 10:55:49.329379 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1412/2283] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:55:49 2022] POST /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c/os-volume_attachments => generated 174 bytes in 97 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 24 10:55:49.334335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:49.334824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:49.340888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.341363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.342115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.342547 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.367314 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:55:49.633907 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-32289d51-b331-40d3-8264-0d3a1c55ad7e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 204 len: 0 microversion: 2.1 time: 0.303042 May 24 10:55:49.634605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1413/2284] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:55:49 2022] DELETE /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) May 24 10:55:49.642783 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:49.643616 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:49.649590 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.650128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.651081 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:49.651510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:49.782786 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:49.790826 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c1c4788f-d8f5-4a1e-bce5-6193cf67463e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1681 microversion: 2.1 time: 0.151251 May 24 10:55:49.791401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1414/2285] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:49 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1681 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:50.808654 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:50.809342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:50.815804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:50.816252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:50.817005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:50.817472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:50.942549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:50.950405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-59b06b50-cb38-47d0-a20e-fcbc3f28a26b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1681 microversion: 2.1 time: 0.146146 May 24 10:55:50.950975 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1415/2286] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:50 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1681 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:51.968847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:51.969700 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:51.976506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:51.977023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:51.978178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:51.978757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:52.047041 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:52.060810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3b481f03-b297-423e-aa09-6fd6e9dc015b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 200 len: 1614 microversion: 2.1 time: 0.095983 May 24 10:55:52.070929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1416/2287] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:51 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 1614 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:55:52.070929 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-2d98a004-0e8f-4c1b-8b7b-78b73255fa61 req-b5ec3c63-bdd0-43ba-860c-694978b8f558 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "319b1d20-c508-46d3-b6af-850a23fd0e8c", "name": "network-vif-deleted", "tag": "bc3cb441-e98a-4708-90a7-614e612f2219"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:55:52.078018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-2d98a004-0e8f-4c1b-8b7b-78b73255fa61 req-b5ec3c63-bdd0-43ba-860c-694978b8f558 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:52.078457 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-2d98a004-0e8f-4c1b-8b7b-78b73255fa61 req-b5ec3c63-bdd0-43ba-860c-694978b8f558 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:52.090732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-2d98a004-0e8f-4c1b-8b7b-78b73255fa61 req-b5ec3c63-bdd0-43ba-860c-694978b8f558 service nova] Creating event network-vif-deleted:bc3cb441-e98a-4708-90a7-614e612f2219 for instance 319b1d20-c508-46d3-b6af-850a23fd0e8c on hv-cinder-84311 May 24 10:55:52.096079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-2d98a004-0e8f-4c1b-8b7b-78b73255fa61 req-b5ec3c63-bdd0-43ba-860c-694978b8f558 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032374 May 24 10:55:52.096814 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1417/2288] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:55:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:55:53.080943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:53.081761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:53.088704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:53.089228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:53.090024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:53.090463 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:53.104567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c HTTP/1.1 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:55:53.106237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:55:53.111125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '319b1d20-c508-46d3-b6af-850a23fd0e8c'} May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:55:53.113519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:55:53.116996 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:55:53.120511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:55:53.120511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cd8e3805-56cd-4090-b784-02109897aa55 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 404 len: 111 microversion: 2.1 time: 0.031642 May 24 10:55:53.120511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1418/2289] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:53 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:55:53.120511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4f06c212-affb-4f3b-8aa8-70897f43ff37 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '.version_select at 0x7f87374d6310>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:53.189810 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4f06c212-affb-4f3b-8aa8-70897f43ff37 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-471115130" status: 202 len: 0 microversion: 2.1 time: 0.076932 May 24 10:55:53.190410 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1419/2290] 10.222.0.35 () {58 vars in 1217 bytes} [Tue May 24 10:55:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-471115130 => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:55:53.958800 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:53.959292 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:53.965849 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:53.966294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:53.967038 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:53.967467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:54.047074 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lazy-loading 'services' on Instance uuid 9405c433-2f5d-4f00-86f5-6b632883f1c6 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:55:54.308367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ec44f2ff-b91b-4f11-94b4-d1626c94d812 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments/506e7d46-e487-4c8b-96c9-e909a582ce64" status: 202 len: 0 microversion: 2.1 time: 0.353447 May 24 10:55:54.313918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1420/2291] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:55:53 2022] DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments/506e7d46-e487-4c8b-96c9-e909a582ce64 => generated 0 bytes in 355 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:55:54.317101 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:54.318159 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:54.325152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:54.325662 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:54.326428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:54.326862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:54.356924 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-01bb6281-0b25-492e-9142-cf6790783259 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043744 May 24 10:55:54.357515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1421/2292] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:55:54 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:55:55.368428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:55.369532 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:55.377586 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:55.378037 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:55.378792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:55.379223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:55.409669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-58cc945c-c656-4ec5-845f-fb3f5b8ad427 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045124 May 24 10:55:55.413444 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1422/2293] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:55:55 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:55:56.421945 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:56.422813 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:56.429310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:56.429769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:56.431356 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:56.431740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:56.460408 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-fa771276-6920-4bf2-baab-4afe80bd071d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042461 May 24 10:55:56.460983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1423/2294] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:55:56 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 24 10:55:56.518084 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:56.518582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:56.525117 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:56.525676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:56.526448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:56.526886 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:56.551597 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:55:56.806347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-327cafcd-e5d5-41c0-b2df-f14399bd192d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 204 len: 0 microversion: 2.1 time: 0.291770 May 24 10:55:56.807012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1424/2295] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:55:56 2022] DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 24 10:55:56.815252 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:56.815952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:56.823019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:56.823471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:56.824245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:56.824680 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:56.958327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:56.966375 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d1b5f734-4a95-44e8-80e6-62ce4c5ea004 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1682 microversion: 2.1 time: 0.155130 May 24 10:55:56.966942 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1425/2296] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:56 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1682 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:57.984241 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:57.984919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:57.991396 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:57.991842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:57.992595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:57.993035 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:57.997119 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:55:58.119954 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:58.127347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-93b912f5-b353-4223-86bb-c4976e3ac1d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1682 microversion: 2.1 time: 0.147054 May 24 10:55:58.127918 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1426/2297] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:57 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1682 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:59.145323 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:55:59.146262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:55:59.152812 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:59.153258 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:59.154104 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:59.154531 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:59.222932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:55:59.231063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-deed40e9-1b30-489c-9eaa-44dfffa6abec tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1614 microversion: 2.1 time: 0.090317 May 24 10:55:59.231847 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1427/2298] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:55:59 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1614 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:55:59.458708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-d7824f48-caf0-4613-a904-6c459fed52e0 req-1f4bfe91-70ef-4fb3-942c-5ef2d32b902f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9405c433-2f5d-4f00-86f5-6b632883f1c6", "name": "network-vif-deleted", "tag": "17e449ec-23f4-46ba-bf62-0a0bb01b298f"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:55:59.467198 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-d7824f48-caf0-4613-a904-6c459fed52e0 req-1f4bfe91-70ef-4fb3-942c-5ef2d32b902f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:55:59.467645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-d7824f48-caf0-4613-a904-6c459fed52e0 req-1f4bfe91-70ef-4fb3-942c-5ef2d32b902f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:55:59.479245 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-d7824f48-caf0-4613-a904-6c459fed52e0 req-1f4bfe91-70ef-4fb3-942c-5ef2d32b902f service nova] Creating event network-vif-deleted:17e449ec-23f4-46ba-bf62-0a0bb01b298f for instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 on hv-cinder-84311 May 24 10:55:59.484653 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-d7824f48-caf0-4613-a904-6c459fed52e0 req-1f4bfe91-70ef-4fb3-942c-5ef2d32b902f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029804 May 24 10:55:59.485255 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1428/2299] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:55:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:56:00.250370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:00.251165 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:00.257809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:00.259228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:00.259228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:00.260419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:00.292980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:00.293471 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:00.293927 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lazy-loading 'fault' on Instance uuid 9405c433-2f5d-4f00-86f5-6b632883f1c6 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:56:00.338064 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:00.345668 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a187e7bd-4b48-4f34-8fc0-ecf514433db0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 200 len: 1331 microversion: 2.1 time: 0.100734 May 24 10:56:00.346259 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1429/2300] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:00 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 1331 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:56:01.363105 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:01.363769 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:01.370350 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:01.370796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:01.371549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:01.371976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:01.385803 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 HTTP/1.1 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:56:01.387401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:56:01.389224 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '9405c433-2f5d-4f00-86f5-6b632883f1c6'} May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:56:01.390345 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:56:01.392145 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:56:01.393780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:56:01.393780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-2a97aa7f-00c2-4d07-acd4-80edb025748b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 404 len: 111 microversion: 2.1 time: 0.029524 May 24 10:56:01.393780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1430/2301] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:01 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:56:01.397031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f8436c90-b6f2-413f-a6b8-2b61ea5504cc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '.version_select at 0x7f87374d6700>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:01.458294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f8436c90-b6f2-413f-a6b8-2b61ea5504cc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-610908888" status: 202 len: 0 microversion: 2.1 time: 0.064677 May 24 10:56:01.458876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1431/2302] 10.222.0.35 () {58 vars in 1217 bytes} [Tue May 24 10:56:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-610908888 => generated 0 bytes in 66 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:56:02.419867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-86bb1a11-a497-42c9-8ed8-4d166f4bb01f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: .version_select at 0x7f8737557670>, body: {"keypair": {"name": "tempest-keypair-1409837217", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAnZrzSfL0C4qNSkhSXeXfnAhNGqITPDRykwMR3LEud6xcQFvlk95Kw1mtTRQy2LBGNzwC2nRqFv4SlybRaiqbaY27MklSeahU9p7yZMt2LSNnWKgubLLw60O+xJ3lUg+w=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:56:02.435530 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-86bb1a11-a497-42c9-8ed8-4d166f4bb01f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:56:02.439346 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-86bb1a11-a497-42c9-8ed8-4d166f4bb01f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:56:02.520932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-86bb1a11-a497-42c9-8ed8-4d166f4bb01f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.104925 May 24 10:56:02.521580 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1432/2303] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:56:02 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:56:04.583414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-592460894", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "28f3e00d-1916-440e-8708-49c75b5dbe5a"}], "security_groups": [{"name": "tempest-securitygroup--738770770"}], "key_name": "tempest-keypair-1409837217", "user_data": "}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:56:04.683118 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] validate_networks() for [('28f3e00d-1916-440e-8708-49c75b5dbe5a', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:56:05.049174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:56:05.049634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:56:05.050113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:56:05.050526 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:56:05.050938 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:56:05.060020 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:05.060465 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:05.061276 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:05.061724 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:05.079446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:56:05.083612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:56:05.095060 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:56:05.097291 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:56:05.201366 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a7ed4033-5964-4056-9503-8b3edf3571fc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.622976 May 24 10:56:05.202016 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1433/2304] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:56:04 2022] POST /compute/v2.1/servers => generated 401 bytes in 624 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 24 10:56:05.212294 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5b260e80-f131-4c44-9cb3-754594bd9aa3 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:05.212973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5b260e80-f131-4c44-9cb3-754594bd9aa3 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:05.266010 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5b260e80-f131-4c44-9cb3-754594bd9aa3 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:05.273285 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5b260e80-f131-4c44-9cb3-754594bd9aa3 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1241 microversion: 2.1 time: 0.064936 May 24 10:56:05.274173 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1434/2305] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:05 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1241 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:06.290169 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:06.290888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:06.297431 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:06.297884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:06.298641 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:06.299097 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:06.360171 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:06.367503 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-a0a6abc0-f934-4843-800c-ca0be7dfafa2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1237 microversion: 2.1 time: 0.081246 May 24 10:56:06.368067 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1435/2306] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:06 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1237 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:07.384936 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:07.385782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:07.391986 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:07.392439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:07.393195 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:07.393824 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:07.456623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:07.465023 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-911670f2-0d04-480f-a80b-a86925f3be09 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1311 microversion: 2.1 time: 0.083998 May 24 10:56:07.465624 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1436/2307] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:07 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1311 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:08.484395 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:08.485124 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:08.488553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:56:08.492018 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:08.492708 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:08.493632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:08.494063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:08.644413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:08.652761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cf92df3c-49f9-481a-89e4-841faca1ca1a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.172189 May 24 10:56:08.653589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1437/2308] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:08 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) May 24 10:56:09.116161 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-8f93d62a-b6b1-4470-879e-a37811ae8f50 req-06f5e73a-04fd-46ab-889c-21cc5dcaa023 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "02d1ec17-4901-461b-8ede-d809e0d782f7", "tag": "8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:56:09.126223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-8f93d62a-b6b1-4470-879e-a37811ae8f50 req-06f5e73a-04fd-46ab-889c-21cc5dcaa023 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:09.129540 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-8f93d62a-b6b1-4470-879e-a37811ae8f50 req-06f5e73a-04fd-46ab-889c-21cc5dcaa023 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:09.139287 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-8f93d62a-b6b1-4470-879e-a37811ae8f50 req-06f5e73a-04fd-46ab-889c-21cc5dcaa023 service nova] Creating event network-changed:8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57 for instance 02d1ec17-4901-461b-8ede-d809e0d782f7 on hv-cinder-84311 May 24 10:56:09.146729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-8f93d62a-b6b1-4470-879e-a37811ae8f50 req-06f5e73a-04fd-46ab-889c-21cc5dcaa023 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037271 May 24 10:56:09.147334 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1438/2309] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:56:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:56:09.677414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:09.678116 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:09.685085 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:09.685559 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:09.686359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:09.686790 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:09.834483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:09.844246 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3073d888-6dc4-4e7c-b325-40580c8f8f5f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.171913 May 24 10:56:09.844851 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1439/2310] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:09 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:10.861228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:10.861952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:10.868318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:10.869467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:10.870310 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:10.870746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:11.000501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:11.007888 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-99dd546c-3051-4621-ae16-dd44fee0e97a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.150630 May 24 10:56:11.008447 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1440/2311] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:10 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:12.023821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:12.024464 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:12.031253 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:12.031703 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:12.032459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:12.032895 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:12.158728 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:12.166481 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1a893b7b-1ff2-44be-baa6-ceeaf284d73f tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.146485 May 24 10:56:12.167047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1441/2312] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:12 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:13.183079 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:13.183686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:13.190174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:13.190620 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:13.191374 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:13.191811 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:13.323947 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:13.331316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-63b658b3-66ed-43c0-b61a-e5db0b626254 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.152228 May 24 10:56:13.331878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1442/2313] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:13 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:14.348151 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:14.348781 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:14.356052 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:14.366029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:14.366029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:14.366029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:14.499909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:14.507651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-79111a7a-c578-4733-9e5c-d157eb2f1cda tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.163302 May 24 10:56:14.508230 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1443/2314] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:14 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:15.524511 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:15.526496 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:15.527243 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:56:15.527941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=12 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:56:15.533210 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:15.534841 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:15.535605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:15.536056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:15.668932 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:15.676179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b3862cbc-635f-4ca0-875c-fafb3433986b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.155487 May 24 10:56:15.676965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1444/2315] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:15 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:15.690003 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-7809d100-391d-4fd6-9d55-b7d363b7f8f1 req-ae4b25ef-ce47-4de8-a76f-40fa2981d097 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02d1ec17-4901-461b-8ede-d809e0d782f7", "name": "network-vif-plugged", "status": "completed", "tag": "8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:56:15.698515 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7809d100-391d-4fd6-9d55-b7d363b7f8f1 req-ae4b25ef-ce47-4de8-a76f-40fa2981d097 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:15.708992 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-7809d100-391d-4fd6-9d55-b7d363b7f8f1 req-ae4b25ef-ce47-4de8-a76f-40fa2981d097 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:15.709816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-7809d100-391d-4fd6-9d55-b7d363b7f8f1 req-ae4b25ef-ce47-4de8-a76f-40fa2981d097 service nova] Creating event network-vif-plugged:8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57 for instance 02d1ec17-4901-461b-8ede-d809e0d782f7 on hv-cinder-84311 May 24 10:56:15.715438 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-7809d100-391d-4fd6-9d55-b7d363b7f8f1 req-ae4b25ef-ce47-4de8-a76f-40fa2981d097 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028957 May 24 10:56:15.716004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1445/2316] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:56:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:56:16.694442 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:16.695146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:16.702424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:16.702885 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:16.703649 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:16.704083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:16.837635 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:16.845183 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d5c142ab-29b5-4d45-8ff2-9f820086e4d6 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1366 microversion: 2.1 time: 0.155488 May 24 10:56:16.845799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1446/2317] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:16 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:17.862012 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:17.862605 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:17.869564 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:17.870013 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:17.870772 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:17.871203 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:17.998579 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:56:18.005979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-007f729f-3bf8-4274-9005-cb87ee67caa0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1557 microversion: 2.1 time: 0.147844 May 24 10:56:18.006549 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1447/2318] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:56:17 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1557 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:56:18.022312 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:18.022794 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:18.028833 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:18.029307 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:18.030047 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:18.030473 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:18.104972 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-27b89367-100b-46ff-941b-26fefb9cb0fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.086149 May 24 10:56:18.105602 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1448/2319] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:56:18 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-interface => generated 285 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:56:19.073387 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-aaa2f0ae-0b17-4592-bec1-7cd794373e34 req-e3927746-01a9-44ab-ac9a-5e92eac2ac8f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "02d1ec17-4901-461b-8ede-d809e0d782f7", "tag": "8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:56:19.081581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-aaa2f0ae-0b17-4592-bec1-7cd794373e34 req-e3927746-01a9-44ab-ac9a-5e92eac2ac8f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:19.082280 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-aaa2f0ae-0b17-4592-bec1-7cd794373e34 req-e3927746-01a9-44ab-ac9a-5e92eac2ac8f service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:19.086329 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=100 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:56:19.093660 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-aaa2f0ae-0b17-4592-bec1-7cd794373e34 req-e3927746-01a9-44ab-ac9a-5e92eac2ac8f service nova] Creating event network-changed:8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57 for instance 02d1ec17-4901-461b-8ede-d809e0d782f7 on hv-cinder-84311 May 24 10:56:19.098576 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-aaa2f0ae-0b17-4592-bec1-7cd794373e34 req-e3927746-01a9-44ab-ac9a-5e92eac2ac8f service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029443 May 24 10:56:19.099166 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1449/2320] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:56:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:56:55.217872 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=24 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:56:55.222320 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:56:55.222896 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:55.228881 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:55.230397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:55.230397 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:55.230647 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:55.254167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-601afcfd-ebc3-41c8-ba16-e7f5a953e319 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/ips" status: 200 len: 150 microversion: 2.1 time: 0.038895 May 24 10:56:55.254738 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1450/2321] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:56:55 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/ips => generated 150 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:56:56.869801 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "49822cd0-8616-47a9-b5af-7cce049ef0b4"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:56:56.871009 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:56:56.878362 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:56.878815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:56.879604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:56:56.880089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:56:57.585043 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-946b21cf-2e7f-4c67-9cca-2511bd28ed27 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.719070 May 24 10:56:57.585686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1451/2322] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:56:56 2022] POST /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments => generated 194 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:57:05.074622 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "49822cd0-8616-47a9-b5af-7cce049ef0b4"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:05.075754 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:05.082317 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:05.082761 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:05.083510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:05.083937 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:05.161632 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Invalid volume: volume 49822cd0-8616-47a9-b5af-7cce049ef0b4 is already attached to instances: 02d1ec17-4901-461b-8ede-d809e0d782f7 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: POST /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments HTTP/1.1 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 74 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:57:05.163446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {"volumeAttachment": {"volumeId": "49822cd0-8616-47a9-b5af-7cce049ef0b4"}} May 24 10:57:05.165088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Invalid volume: volume 49822cd0-8616-47a9-b5af-7cce049ef0b4 is already attached to instances: 02d1ec17-4901-461b-8ede-d809e0d782f7 May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'server_id': '02d1ec17-4901-461b-8ede-d809e0d782f7', 'body': {'volumeAttachment': {'volumeId': '49822cd0-8616-47a9-b5af-7cce049ef0b4'}}} May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 49822cd0-8616-47a9-b5af-7cce049ef0b4 is already attached to instances: 02d1ec17-4901-461b-8ede-d809e0d782f7 May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:05.166717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 49822cd0-8616-47a9-b5af-7cce049ef0b4 is already attached to instances: 02d1ec17-4901-461b-8ede-d809e0d782f7 May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:57:05.168123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 49822cd0-8616-47a9-b5af-7cce049ef0b4 is already attached to instances: 02d1ec17-4901-461b-8ede-d809e0d782f7 May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 400 to user: Invalid volume: volume 49822cd0-8616-47a9-b5af-7cce049ef0b4 is already attached to instances: 02d1ec17-4901-461b-8ede-d809e0d782f7 {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-966b5936-79fd-44c9-8d7b-2ae88992cbcd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.095041 May 24 10:57:05.170090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1452/2323] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:57:05 2022] POST /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments => generated 174 bytes in 96 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) May 24 10:57:05.225414 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:05.225842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:05.231893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:05.232335 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:05.233290 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:05.233726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:05.313978 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lazy-loading 'services' on Instance uuid 02d1ec17-4901-461b-8ede-d809e0d782f7 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:57:05.482514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b7e1329f-1966-41a5-98c7-24b685d4a83e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments/49822cd0-8616-47a9-b5af-7cce049ef0b4" status: 202 len: 0 microversion: 2.1 time: 0.260786 May 24 10:57:05.483614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1453/2324] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:57:05 2022] DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments/49822cd0-8616-47a9-b5af-7cce049ef0b4 => generated 0 bytes in 263 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:57:05.492247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:05.493313 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:05.500365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:05.501058 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:05.501607 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:05.502039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:05.531054 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-6ec8099a-9968-4d3a-9f46-3c7db2336dea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042936 May 24 10:57:05.531615 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1454/2325] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:57:05 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:57:06.542490 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:06.543413 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:06.550909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:06.556585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:06.556585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:06.556585 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:06.582683 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b5d52205-b464-4105-bddf-31593b652883 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043964 May 24 10:57:06.585315 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1455/2326] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:57:06 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 24 10:57:06.642685 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:06.647462 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:06.649211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:06.649690 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:06.650448 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:06.650877 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:06.677141 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:57:06.970136 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-d86763e2-8e23-42e8-9b41-8c40c8e405a0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 204 len: 0 microversion: 2.1 time: 0.331134 May 24 10:57:06.970822 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1456/2327] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:57:06 2022] DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 24 10:57:06.985518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:06.985518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:06.986487 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:06.986941 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:06.987744 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:06.988175 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:07.122568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:07.130704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-75f705f0-3022-4f7c-9bc4-7e0c5e650d01 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1681 microversion: 2.1 time: 0.155462 May 24 10:57:07.131265 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1457/2328] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:06 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:08.147706 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:08.148363 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:08.154737 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:08.155179 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:08.155930 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:08.156359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:08.283469 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:08.291155 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-74db4242-5f77-453e-a83b-a0bb6bb14cb7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1681 microversion: 2.1 time: 0.147285 May 24 10:57:08.291726 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1458/2329] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:08 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1681 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:09.308188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:09.308860 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:09.315798 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:09.316247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:09.317005 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:09.317472 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:09.617943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:09.626148 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3e811b16-0606-44c0-a7b1-3cbf41449009 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1681 microversion: 2.1 time: 0.321778 May 24 10:57:09.627075 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1459/2330] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:09 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1681 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:09.925676 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-5d602c67-34e8-4d31-94ed-f949ece419ce req-7fcfe80b-3aeb-4cca-96e4-1a5d8c26f5ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02d1ec17-4901-461b-8ede-d809e0d782f7", "name": "network-vif-deleted", "tag": "8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:09.933645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5d602c67-34e8-4d31-94ed-f949ece419ce req-7fcfe80b-3aeb-4cca-96e4-1a5d8c26f5ce service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:09.937034 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-5d602c67-34e8-4d31-94ed-f949ece419ce req-7fcfe80b-3aeb-4cca-96e4-1a5d8c26f5ce service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:09.945773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-5d602c67-34e8-4d31-94ed-f949ece419ce req-7fcfe80b-3aeb-4cca-96e4-1a5d8c26f5ce service nova] Creating event network-vif-deleted:8c48931d-7f2c-49e1-9ab0-6d5ed03b2e57 for instance 02d1ec17-4901-461b-8ede-d809e0d782f7 on hv-cinder-84311 May 24 10:57:09.950582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-5d602c67-34e8-4d31-94ed-f949ece419ce req-7fcfe80b-3aeb-4cca-96e4-1a5d8c26f5ce service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029158 May 24 10:57:09.951146 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1460/2331] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:57:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:57:10.645663 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:10.646501 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:10.654991 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:10.655440 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:10.656212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:10.656675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:10.698619 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:10.699056 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:10.699525 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lazy-loading 'fault' on Instance uuid 02d1ec17-4901-461b-8ede-d809e0d782f7 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:57:10.754427 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:10.763127 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-9e206515-377e-4e47-a8d7-69ec02ede74b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 200 len: 1331 microversion: 2.1 time: 0.123360 May 24 10:57:10.763720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1461/2332] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:10 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 1331 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:11.781510 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:11.782891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:11.789228 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:11.789729 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:11.790486 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:11.790919 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:11.804550 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 HTTP/1.1 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:57:11.806216 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:57:11.808123 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '02d1ec17-4901-461b-8ede-d809e0d782f7'} May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:57:11.809188 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:57:11.810958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:57:11.812508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:57:11.812508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7b1f48eb-279c-4005-98ad-1dc56b4ef5d9 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 404 len: 111 microversion: 2.1 time: 0.029976 May 24 10:57:11.812508 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1462/2333] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:11 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:57:11.816416 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-406928ec-80c8-48ff-9934-bac68c0aef9c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '.version_select at 0x7f87320b71f0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:11.885639 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-406928ec-80c8-48ff-9934-bac68c0aef9c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1409837217" status: 202 len: 0 microversion: 2.1 time: 0.072995 May 24 10:57:11.886214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1463/2334] 10.222.0.35 () {58 vars in 1220 bytes} [Tue May 24 10:57:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1409837217 => generated 0 bytes in 74 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) May 24 10:57:12.600025 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-db4fd35a-a240-4200-8d91-553b78346e31 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: .version_select at 0x7f8737509310>, body: {"keypair": {"name": "tempest-keypair-1957006250", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCE4dxi9+IJEoRaFc/AgdWpFh6P095DRxv0MWjebBljxyQqz9ThjjDxLlFN6A8MQfxm+9Q6LdSUgSxjzwH9cgUHjnu/x7zl0Iq6DH1OqLpQFd7ysVZAuV6vzQ7F5xUaA9g=="}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:12.614696 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-db4fd35a-a240-4200-8d91-553b78346e31 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:57:12.618518 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-db4fd35a-a240-4200-8d91-553b78346e31 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'key_pairs'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:57:12.690804 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-db4fd35a-a240-4200-8d91-553b78346e31 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.094648 May 24 10:57:12.697059 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1464/2335] 10.222.0.35 () {60 vars in 1158 bytes} [Tue May 24 10:57:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:57:14.654373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1504170626", "imageRef": "1627b27e-e8d1-48c8-a446-6d8b74297155", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "28f3e00d-1916-440e-8708-49c75b5dbe5a"}], "security_groups": [{"name": "tempest-securitygroup--1549747877"}], "key_name": "tempest-keypair-1957006250", "user_data": "}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:14.754190 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.network.neutron [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] validate_networks() for [('28f3e00d-1916-440e-8708-49c75b5dbe5a', None, None, None, None, None)] {{(pid=118232) validate_networks /opt/stack/nova/nova/network/neutron.py:2609}} May 24 10:57:15.116133 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} May 24 10:57:15.116567 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image limits 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} May 24 10:57:15.117045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Flavor pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} May 24 10:57:15.117484 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Image pref 0:0:0 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} May 24 10:57:15.117897 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.virt.hardware [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=118232) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} May 24 10:57:15.126768 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:15.127212 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:15.128006 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:15.128436 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:15.145979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:393}} May 24 10:57:15.149988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.quota [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Getting quotas for user ea49ebe4f80d48399086cacfe7b4bfdb and project c94763dacd084cd69ee0729ee21d28a2. Resources: {'cores', 'instances', 'ram'} {{(pid=118232) _get_quotas /opt/stack/nova/nova/quota.py:383}} May 24 10:57:15.161545 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Going to run 1 instances... {{(pid=118232) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} May 24 10:57:15.163796 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1627b27e-e8d1-48c8-a446-6d8b74297155',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=118232) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} May 24 10:57:15.319204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-642f25ac-fd49-4872-b1bf-35953315ffe7 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.669568 May 24 10:57:15.319842 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1465/2336] 10.222.0.35 () {60 vars in 1146 bytes} [Tue May 24 10:57:14 2022] POST /compute/v2.1/servers => generated 402 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) May 24 10:57:15.330042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-897bd824-fe01-4db9-b0f4-16d344131e3d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:15.330640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-897bd824-fe01-4db9-b0f4-16d344131e3d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:15.382845 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-897bd824-fe01-4db9-b0f4-16d344131e3d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:15.389439 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-897bd824-fe01-4db9-b0f4-16d344131e3d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1242 microversion: 2.1 time: 0.063663 May 24 10:57:15.390021 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1466/2337] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:15 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1242 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:16.407218 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:16.407887 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:16.415834 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:16.416196 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:16.416956 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:16.417415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:16.480354 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:16.483808 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:57:16.488751 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8dd1d0c4-aca0-4032-84e1-ca94e9f8624b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1238 microversion: 2.1 time: 0.085497 May 24 10:57:16.489318 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1467/2338] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:16 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1238 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:17.506591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:17.507308 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:17.514223 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:17.514675 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:17.515446 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:17.515878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:17.584283 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:17.591669 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-b16a0dfd-1bdd-45f7-b935-6c8c2622d5df tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1300 microversion: 2.1 time: 0.090378 May 24 10:57:17.592553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1468/2339] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:17 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1300 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:18.616516 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:18.617326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:18.624262 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:18.624983 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:18.625785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:18.626219 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:18.766109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:18.774855 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-913e4e66-12cb-4645-8b01-2c6638aa708d tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.162089 May 24 10:57:18.775884 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1469/2340] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:18 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:18.781467 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-e608c0d3-fb51-4a7e-a69c-2743fa8c3153 req-65557a40-95f0-4321-b9ae-5fba579cba47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6cfa310-aa7a-4770-b439-8c7fc69a5ac9", "tag": "3b82cd23-20b3-4376-9415-3d0cdf98624e"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:18.791958 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-e608c0d3-fb51-4a7e-a69c-2743fa8c3153 req-65557a40-95f0-4321-b9ae-5fba579cba47 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:18.792558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-e608c0d3-fb51-4a7e-a69c-2743fa8c3153 req-65557a40-95f0-4321-b9ae-5fba579cba47 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:18.806730 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-e608c0d3-fb51-4a7e-a69c-2743fa8c3153 req-65557a40-95f0-4321-b9ae-5fba579cba47 service nova] Creating event network-changed:3b82cd23-20b3-4376-9415-3d0cdf98624e for instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 on hv-cinder-84311 May 24 10:57:18.812125 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-e608c0d3-fb51-4a7e-a69c-2743fa8c3153 req-65557a40-95f0-4321-b9ae-5fba579cba47 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034513 May 24 10:57:18.813638 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1470/2341] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:57:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:57:19.793069 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:19.793820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:19.801425 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:19.801880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:19.802637 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:19.803073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:19.933553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:19.941612 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f6ae94ad-eef7-4338-b2d6-d05ba6ed7ef5 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.152716 May 24 10:57:19.942184 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1471/2342] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:19 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:20.959537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:20.960214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:20.966646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:20.967094 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:20.969694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:20.969694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:21.097483 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:21.104878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-cfcf0f06-0d69-4d79-b505-fce6137d0e6e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.150220 May 24 10:57:21.105704 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1472/2343] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:20 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:22.122656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:22.123337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:22.129720 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:22.130167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:22.130923 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:22.131355 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:22.259539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:22.266862 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-57b363a0-bce2-492f-a35d-dce508fb12fe tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.148236 May 24 10:57:22.267424 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1473/2344] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:22 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:23.283830 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:23.284459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:23.290946 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:23.291398 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:23.292156 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:23.292591 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:23.420019 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:23.427614 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-dde0bf04-648a-4b71-ba46-f2d4d3479650 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.147702 May 24 10:57:23.428191 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1474/2345] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:23 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:24.034137 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-223f5be9-a76f-4a9c-a717-d3e5c6ce2ad6 req-ab3e6f43-d84e-4fd2-9119-94c0a5234130 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6cfa310-aa7a-4770-b439-8c7fc69a5ac9", "name": "network-vif-plugged", "status": "completed", "tag": "3b82cd23-20b3-4376-9415-3d0cdf98624e"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:24.042163 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-223f5be9-a76f-4a9c-a717-d3e5c6ce2ad6 req-ab3e6f43-d84e-4fd2-9119-94c0a5234130 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:24.046795 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-223f5be9-a76f-4a9c-a717-d3e5c6ce2ad6 req-ab3e6f43-d84e-4fd2-9119-94c0a5234130 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:24.054359 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-223f5be9-a76f-4a9c-a717-d3e5c6ce2ad6 req-ab3e6f43-d84e-4fd2-9119-94c0a5234130 service nova] Creating event network-vif-plugged:3b82cd23-20b3-4376-9415-3d0cdf98624e for instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 on hv-cinder-84311 May 24 10:57:24.059237 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-223f5be9-a76f-4a9c-a717-d3e5c6ce2ad6 req-ab3e6f43-d84e-4fd2-9119-94c0a5234130 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029650 May 24 10:57:24.060029 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1475/2346] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:57:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:57:24.444656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:24.445373 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:24.451959 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:24.452401 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:24.453150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:24.453717 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:24.583286 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:24.590686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5e32b75d-b790-4f01-8c85-67a8671a851e tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.150282 May 24 10:57:24.591297 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1476/2347] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:24 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:25.607757 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:25.609111 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:25.611157 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=2 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:57:25.611893 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell0:SELECT=12 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:57:25.613645 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=87,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:57:25.618476 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:25.618926 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:25.619692 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:25.620128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:25.751740 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:25.759186 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c2a44e29-48a7-4120-ba77-6eaf6ead9264 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1368 microversion: 2.1 time: 0.155247 May 24 10:57:25.759753 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1477/2348] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:25 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1368 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:26.776168 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:26.776900 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:26.783519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:26.783976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:26.784815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:26.785247 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:26.914367 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:57:26.922128 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-1acee126-3ccb-4bf7-88cc-31ddc5e0fd23 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1558 microversion: 2.1 time: 0.150136 May 24 10:57:26.922691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1478/2349] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:57:26 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1558 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:57:26.939109 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:57:26.939732 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:57:26.946459 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:26.946902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:26.947656 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:26.948113 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:27.027178 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8e037f12-b5c2-40ce-9b1d-925b4fefe59b tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091736 May 24 10:57:27.027878 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1479/2350] 10.222.0.35 () {58 vars in 1274 bytes} [Tue May 24 10:57:26 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:57:27.974844 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-3fb457c4-5905-46e3-be0b-10ece785e4ed req-efdd8f2c-7ce1-4e46-8384-80b521fdb7dd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6cfa310-aa7a-4770-b439-8c7fc69a5ac9", "tag": "3b82cd23-20b3-4376-9415-3d0cdf98624e"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:57:27.983750 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-3fb457c4-5905-46e3-be0b-10ece785e4ed req-efdd8f2c-7ce1-4e46-8384-80b521fdb7dd service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:57:27.984275 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-3fb457c4-5905-46e3-be0b-10ece785e4ed req-efdd8f2c-7ce1-4e46-8384-80b521fdb7dd service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:57:27.995651 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-3fb457c4-5905-46e3-be0b-10ece785e4ed req-efdd8f2c-7ce1-4e46-8384-80b521fdb7dd service nova] Creating event network-changed:3b82cd23-20b3-4376-9415-3d0cdf98624e for instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 on hv-cinder-84311 May 24 10:57:28.000686 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-3fb457c4-5905-46e3-be0b-10ece785e4ed req-efdd8f2c-7ce1-4e46-8384-80b521fdb7dd service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030108 May 24 10:57:28.001404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1480/2351] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:57:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:58:04.159167 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_api:SELECT=7 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:04.159909 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG dbcounter [-] [118232] Writing DB stats nova_cell1:SELECT=64 {{(pid=118232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:04.164152 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:04.164734 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:04.170891 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:04.175053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:04.175053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:04.175053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:04.195289 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5a720c05-218e-411f-a240-cc0d227d87d8 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/ips" status: 200 len: 149 microversion: 2.1 time: 0.037963 May 24 10:58:04.195852 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1481/2352] 10.222.0.35 () {58 vars in 1247 bytes} [Tue May 24 10:58:04 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/ips => generated 149 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:58:05.749979 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ce5edc14-afb3-422a-92e6-8bb59accbf60"}} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:58:05.751042 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:05.757780 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:05.758229 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:05.758989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:05.759423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:06.640707 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-e5f45497-9345-428e-89b7-3a4ad36ed9cb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "POST /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.895218 May 24 10:58:06.641319 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1482/2353] 10.222.0.35 () {60 vars in 1322 bytes} [Tue May 24 10:58:05 2022] POST /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments => generated 194 bytes in 897 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:58:14.227214 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:14.227809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:14.235192 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:14.235636 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:14.236389 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:14.236821 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:14.317644 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.objects.instance [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lazy-loading 'services' on Instance uuid f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 {{(pid=118232) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} May 24 10:58:14.533417 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f86215c0-0e2d-42f5-a0a1-e1320e3b3805 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments/ce5edc14-afb3-422a-92e6-8bb59accbf60" status: 202 len: 0 microversion: 2.1 time: 0.310199 May 24 10:58:14.534048 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1483/2354] 10.222.0.35 () {58 vars in 1415 bytes} [Tue May 24 10:58:14 2022] DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments/ce5edc14-afb3-422a-92e6-8bb59accbf60 => generated 0 bytes in 312 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:58:14.542693 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:14.543791 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:14.550672 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:14.551135 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:14.551898 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:14.552337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:14.582799 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-395d4e7a-9a35-4f70-9a7c-c8845d303359 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043972 May 24 10:58:14.583368 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1484/2355] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:58:14 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) May 24 10:58:15.595522 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:15.596449 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:15.602973 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:15.603423 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:15.604189 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:15.604623 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:15.634474 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-db51e8eb-9309-409a-b169-97f8fb7bb97c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043824 May 24 10:58:15.635044 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1485/2356] 10.222.0.35 () {58 vars in 1301 bytes} [Tue May 24 10:58:15 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) May 24 10:58:15.693534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:15.694031 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:15.700004 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:15.700450 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:15.701773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:15.702357 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:15.727045 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Going to try to terminate instance {{(pid=118232) delete /opt/stack/nova/nova/compute/api.py:2664}} May 24 10:58:15.985404 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-72e548cb-8d82-4b29-a600-a1d52f499024 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 204 len: 0 microversion: 2.1 time: 0.295220 May 24 10:58:15.985916 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1486/2357] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:58:15 2022] DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) May 24 10:58:15.998541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:15.999407 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:16.005324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:16.005815 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:16.006558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:16.006988 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:16.358506 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:58:16.365963 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-62e21e52-c0d8-4879-9c27-42b67ff7eefc tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1682 microversion: 2.1 time: 0.371825 May 24 10:58:16.366534 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1487/2358] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:15 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1682 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:58:17.383640 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:17.384316 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:17.391352 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:17.391802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:17.392558 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:17.392989 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:17.519961 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:58:17.527418 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8ad59800-1c68-4aec-9316-860aec29fb9a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1682 microversion: 2.1 time: 0.147635 May 24 10:58:17.527980 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1488/2359] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:17 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1682 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:58:18.543985 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:18.545215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:18.552665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:18.553114 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:18.553908 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:18.554341 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:18.617500 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.policy [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ea49ebe4f80d48399086cacfe7b4bfdb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c94763dacd084cd69ee0729ee21d28a2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=118232) authorize /opt/stack/nova/nova/policy.py:199}} May 24 10:58:18.626399 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c7463d2d-e9a1-458d-8e2e-890ebaaa6c2c tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 200 len: 1614 microversion: 2.1 time: 0.086290 May 24 10:58:18.627204 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1489/2360] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:18 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 1614 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) May 24 10:58:18.640380 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [req-43d07738-db14-48a6-b9dd-cba9f0e9936d req-ae48da8d-46cc-4e12-b948-09fcb16e9965 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6cfa310-aa7a-4770-b439-8c7fc69a5ac9", "name": "network-vif-deleted", "tag": "3b82cd23-20b3-4376-9415-3d0cdf98624e"}]} {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} May 24 10:58:18.648458 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-43d07738-db14-48a6-b9dd-cba9f0e9936d req-ae48da8d-46cc-4e12-b948-09fcb16e9965 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:18.648911 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [req-43d07738-db14-48a6-b9dd-cba9f0e9936d req-ae48da8d-46cc-4e12-b948-09fcb16e9965 service nova] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:18.660088 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.compute.server_external_events [req-43d07738-db14-48a6-b9dd-cba9f0e9936d req-ae48da8d-46cc-4e12-b948-09fcb16e9965 service nova] Creating event network-vif-deleted:3b82cd23-20b3-4376-9415-3d0cdf98624e for instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 on hv-cinder-84311 May 24 10:58:18.665090 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [req-43d07738-db14-48a6-b9dd-cba9f0e9936d req-ae48da8d-46cc-4e12-b948-09fcb16e9965 service nova] 10.222.0.35 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028319 May 24 10:58:18.665705 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1490/2361] 10.222.0.35 () {64 vars in 1317 bytes} [Tue May 24 10:58:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) May 24 10:58:19.645428 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:19.646138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:19.652562 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:19.653014 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:19.653809 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:19.655030 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:19.671749 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 HTTP/1.1 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:19.674039 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:19.676061 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'f6cfa310-aa7a-4770-b439-8c7fc69a5ac9'} May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:19.677110 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:19.678820 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:19.680393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:19.680393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-8d53fec0-8a1f-46ff-a98f-fb2893f73b21 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 404 len: 111 microversion: 2.1 time: 0.034327 May 24 10:58:19.680393 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1491/2362] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:19 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:58:19.683568 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-f1e029c9-b106-4d81-afa7-bcda0c122818 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '.version_select at 0x7f873749a1f0>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:19.775180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-f1e029c9-b106-4d81-afa7-bcda0c122818 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1957006250" status: 202 len: 0 microversion: 2.1 time: 0.095281 May 24 10:58:19.775697 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1492/2363] 10.222.0.35 () {58 vars in 1220 bytes} [Tue May 24 10:58:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1957006250 => generated 0 bytes in 96 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) May 24 10:58:21.738405 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:21.739028 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:21.746342 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:21.746782 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:21.747552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:21.747976 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:21.760934 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 HTTP/1.1 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:21.762539 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.764194 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'f6cfa310-aa7a-4770-b439-8c7fc69a5ac9'} May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:21.765664 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.767065 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.768523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:21.768523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-3079bd9e-c88b-40e5-810f-65038f3dfc59 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 404 len: 111 microversion: 2.1 time: 0.029321 May 24 10:58:21.768523 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1493/2364] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:58:21 2022] DELETE /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:58:21.777837 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:21.778415 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: f6cfa310-aa7a-4770-b439-8c7fc69a5ac9] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:21.784999 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:21.785477 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:21.786220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:21.786646 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:21.799322 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 HTTP/1.1 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:21.800792 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.802595 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': 'f6cfa310-aa7a-4770-b439-8c7fc69a5ac9'} May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:21.803670 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. May 24 10:58:21.805519 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:21.807076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:21.807076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-c9b263e4-999c-4ec3-9e78-1620476806fb tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9" status: 404 len: 111 microversion: 2.1 time: 0.027866 May 24 10:58:21.807076 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1494/2365] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:21 2022] GET /compute/v2.1/servers/f6cfa310-aa7a-4770-b439-8c7fc69a5ac9 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:58:23.244370 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:23.244944 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:23.251731 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.252181 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.252953 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.253419 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.267553 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 HTTP/1.1 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:23.269083 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.270876 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '02d1ec17-4901-461b-8ede-d809e0d782f7'} May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:23.271902 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.273365 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.274785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:23.274785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-aa8ce266-ce26-409f-8baf-c629b7625749 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 404 len: 111 microversion: 2.1 time: 0.029938 May 24 10:58:23.274785 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1495/2366] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:58:23 2022] DELETE /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:58:23.278337 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:23.278943 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 02d1ec17-4901-461b-8ede-d809e0d782f7] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:23.285634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.286089 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.286850 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.287288 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.300347 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 HTTP/1.1 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:23.301866 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.303634 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '02d1ec17-4901-461b-8ede-d809e0d782f7'} May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:23.304713 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. May 24 10:58:23.306394 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.307880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 02d1ec17-4901-461b-8ede-d809e0d782f7 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:23.307880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-7fe1aea7-bbc4-424b-815e-6182e875430a tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7" status: 404 len: 111 microversion: 2.1 time: 0.028362 May 24 10:58:23.307880 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1496/2367] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:23 2022] GET /compute/v2.1/servers/02d1ec17-4901-461b-8ede-d809e0d782f7 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) May 24 10:58:23.310867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:23.311537 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:23.317205 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.317691 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.318445 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.318883 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.332952 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c HTTP/1.1 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:23.334479 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.336220 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '319b1d20-c508-46d3-b6af-850a23fd0e8c'} May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:23.337326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.339053 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.340816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:23.340816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-49ecf5d8-782e-4bf4-955a-a0430a51d8ea tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 404 len: 111 microversion: 2.1 time: 0.028419 May 24 10:58:23.340816 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1497/2368] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:58:23 2022] DELETE /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:58:23.344581 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:23.345215 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 319b1d20-c508-46d3-b6af-850a23fd0e8c] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:23.350873 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.351324 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.352078 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:23.352514 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:23.365694 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c HTTP/1.1 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:23.367174 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.368940 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '319b1d20-c508-46d3-b6af-850a23fd0e8c'} May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:23.369964 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. May 24 10:58:23.371665 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:23.373091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 319b1d20-c508-46d3-b6af-850a23fd0e8c could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:23.373091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-4ac20bf2-0435-4c26-8fa4-aa47d2ae19d0 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c" status: 404 len: 111 microversion: 2.1 time: 0.028563 May 24 10:58:23.373091 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1498/2369] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:23 2022] GET /compute/v2.1/servers/319b1d20-c508-46d3-b6af-850a23fd0e8c => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:58:24.654134 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:24.654739 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:24.661718 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:24.662164 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:24.662917 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:24.663348 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:24.676746 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 HTTP/1.1 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:24.678299 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.680063 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '9405c433-2f5d-4f00-86f5-6b632883f1c6'} May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:24.681227 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.682965 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.684582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:24.684582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-5c8840c7-c6c7-4a9f-bb9d-5dba0ddefef2 tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 404 len: 111 microversion: 2.1 time: 0.029423 May 24 10:58:24.684582 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1499/2370] 10.222.0.35 () {58 vars in 1238 bytes} [Tue May 24 10:58:24 2022] DELETE /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 10:58:24.687589 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Calling method '>' {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} May 24 10:58:24.688211 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.compute.api [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] [instance: 9405c433-2f5d-4f00-86f5-6b632883f1c6] Fetching instance by UUID {{(pid=118232) get /opt/stack/nova/nova/compute/api.py:2920}} May 24 10:58:24.693875 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:24.694327 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:24.695073 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} May 24 10:58:24.695502 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG oslo_concurrency.lockutils [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Lock "0b3c262b-14d7-4ecd-824a-690cdc4a9b4a" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=118232) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} May 24 10:58:24.709326 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.wsgi [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] HTTP exception thrown: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Request method failure captured: May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: request: GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 HTTP/1.1 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept: application/json May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Accept-Encoding: identity May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Connection: close May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Length: 0 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Content-Type: application/json May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Host: 10.222.0.35 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Openstack-System-Scope: None May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: User-Agent: python-urllib3/1.26.9 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Auth-Token: gAAAAABijLlAm4f22c7Sk1yTJSAnxjvgwSRZASQ_EWtTljG_CN8gYGfJS9EXSxXKhmhjLSDb4MiPL03SrwVdLoXiH6tG3UKbkHrvZWs8_efwz_ntDDNGq3udL5RUeQ5UoXkqryS5wERuD776BKiBI4JIGDppS7vrmwVYvC4ui8-WjkSmk1k1hU8 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Id: None May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Domain-Name: None May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Identity-Status: Confirmed May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Is-Admin-Project: True May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Id: default May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Domain-Name: Default May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Project-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Role: member,reader May 24 10:58:24.710802 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Roles: member,reader May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.35:8080", "publicURL": "http://10.222.0.35:8080/v1/AUTH_c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/volume/v3/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2/c94763dacd084cd69ee0729ee21d28a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.35:9696/"}]}] May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Id: c94763dacd084cd69ee0729ee21d28a2 May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-805432205 May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Id: default May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Domain-Name: Default May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Id: ea49ebe4f80d48399086cacfe7b4bfdb May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: X-User-Name: tempest-AttachVolumeNegativeTest-805432205-project May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: method: > May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: exception: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.712577 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: action_args: {'id': '9405c433-2f5d-4f00-86f5-6b632883f1c6'} May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: {{(pid=118232) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) May 24 10:58:24.713773 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = self._get_server( May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. May 24 10:58:24.715541 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: ERROR nova.api.openstack.wsgi May 24 10:58:24.717024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: DEBUG nova.api.openstack.wsgi [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] Returning 404 to user: Instance 9405c433-2f5d-4f00-86f5-6b632883f1c6 could not be found. {{(pid=118232) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} May 24 10:58:24.717024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO nova.api.openstack.requestlog [None req-ee1fd3c9-ed66-47ad-a779-1751a23cc3fd tempest-AttachVolumeNegativeTest-805432205 tempest-AttachVolumeNegativeTest-805432205-project] 10.222.0.35 "GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6" status: 404 len: 111 microversion: 2.1 time: 0.028212 May 24 10:58:24.717024 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: [pid: 118232|app: 0|req: 1500/2371] 10.222.0.35 () {58 vars in 1235 bytes} [Tue May 24 10:58:24 2022] GET /compute/v2.1/servers/9405c433-2f5d-4f00-86f5-6b632883f1c6 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) May 24 11:00:26.362604 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 11:00:26.363180 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 11:00:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 11:03:46.372115 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 11:03:46.372719 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 11:03:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 11:03:49.965150 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 11:03:49.965867 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 11:03:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! May 24 11:03:57.442552 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer May 24 11:03:57.443138 dv-cinder-843118-2-iscsi devstack@n-api.service[118232]: Tue May 24 11:03:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!